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Executive Summary 


Today’s engineering graduates wiil solve tomorrow’s problems in a world that is advancing 
faster and facing more critical challenges than ever before. This situation creates significant 
demand for engineering education to evolve in order to effectively prepare a diverse 
community of engineers for these challenges. Such concerns have led to the publication of 
visionary reports that help orient the work of those committed to the success of engineering 
education. Research in engineering education is central to all of these visions. 

The Need 

Research on the student experience is fundamental to informing the evolution of engineering 
education. A broad understanding of the engineering student experience involves thinking 
about diverse academic pathways, navigation of these pathways, and decision points— how 
students choose engineering programs, navigate through their programs, and then move on 
to jobs and careers. Further, looking at students’ experiences broadly entails not Just 
thinking about their learning (/.e., skill and knowledge development in both technical and 
professional areas) but also their motivation, their identification with engineering, their 
confidence, and their choices after graduation. 

In actuality, there is not one singular student experience, but rather many experiences. 
Research on engineering student experiences can look into systematic differences across 
demographics, disciplines, and campuses; gain insight into the experiences of 
underrepresented students; and create a rich portrait of how students change from first year 
through graduation. Such a broad understanding of the engineering student experience can 
serve as inspiration for designing innovative curricular experiences that support the many 
and varied pathways that students take on their way to becoming an engineer. 

However, an understanding of the engineering student experience is clearly not enough to 
create innovation in engineering education. We need educators who are capable of using 
the research on the student experience. This involves not only preparing tomorrow’s 
educators with conceptions of teaching that enable innovation but also understanding how 
today’s educators make teaching decisions. We also need to be concerned about creating 
the capacity to do such research— in short, we need more researchers. One promising 
approach is to work with educators who are interested in engaging in research, supporting 
them as they negotiate the space between their current activities and their new work in 
engineering education research. To fully support this process, we must also investigate what 
is required for educators to engage in such a path. 

The Center 

The Center for the Advancement of Engineering Education (CAEE) began research in January 
2003 as one of two national higher-education Centers for Teaching and Learning funded by 
the National Science Foundation that year. Two divisions of the NSF provided support: 
Engineering Education and Centers (Engineering Directorate) and the Division of 
Undergraduate Education (Education & Human Resources Directorate). Originally funded for 
2003-2007, supplementary funds from the Engineering Directorate allowed additional 
analysis and dissemination to continue through 2010. 


2 


Enabling Engineering Student Success 


This report describes the work of CAEE— work that addresses the issues highiighted above. 
CAEE engaged in four threads of activity: 

• Academic Pathways Study (APS, 2003-2010) 

• Studies of Engineering Educator Decisions (SEED, 2006-2010) 

• Engineering Teaching Portfoiio Program (ETPP, 2003-2006) 

• institute for Schoiarship on Engineering Education (iSEE, 2003-2008) 

These activities aii invoived an emphasis on the peopie in the engineering education system: 
students, educators, and researchers. The Center activities invoived concurrent and 
interwoven emphasis on both research and capacity buiiding. For exampie, whiie the first 
two efforts (APS and SEED) focused on research and the second two (ETPP and iSEE) were 
primariiy capacity-buiiding efforts, significant capacity-buiiding outcomes were part of the 
first two efforts, and the iast two efforts addressed important research questions. Finaiiy, 
our activities were not oniy motivated by a desire to support future innovation, but 
themseives invoived innovation— from the scaie of the Academic Pathways Study, to the 
noveity of the Studies of Engineering Educator Decisions, to the emphasis on diversity in the 
Engineering Teaching Portfoiio Program, and the fiexibiiity of the institute modei. 

Beiow, we summarize CAEE’s findings and outcomes, foiiowed by highiights from our efforts 
to disseminate the resuits, inciuding a set of research instruments and other materiais that 
are avaiiabie for use by others. We conciude with a iook ahead at next steps and some 
questions for future research. 

The Learning Experiences of Undergraduate Engineering Students: 

The Academic Pathways Study (APS) 

The primary goai of the Academic Pathways Study was to create a rich and wide-ranging 
portrait of the undergraduate engineering iearning experience, using muitipie research 
methods and reiying on the students’ own words for much of the data. Specific research 
questions focused on four areas: 

• Skills: How do students’ engineering design skiiis and understanding of engineering 
practice deveiop and/or change overtime? 

• Identity: How do students come to identify themseives as engineers? How do these 
identities change as they navigate their education? 

• Education: What elements of students’ engineering educations contribute to changes 
examined in the skills and identity questions above? 

• Workplace: How do students conceive of their careers? What skills do early-career 
engineers need as they enter the workplace? 

The Academic Pathways Study addressed these questions with a large, multi-faceted 
research effort that generated a broad and varied set of results. To summarize, the various 
components of the APS included the following: 

• over 5,400 students from around the country 

• multiple research methods (both quantitative and qualitative), including surveys, 
structured and semi-structured (ethnographic) interviews, engineering design tasks, 
and focus groups 

• a four-year longitudinal study at four institutions 
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• a broad, national survey at 21 institutions 

• an additionai coliaboration with Nationai Survey of Student Engagement (NSSE) 
researchers that enabied a broad comparison of engineering undergraduates with 
those in other majors {N > 11,000) 

• a study of over 100 newly hired engineering graduates and 15 of their managers at 
14 companies and organizations 

In this report, we provide not oniy details of the APS findings, but aiso information about the 
scope of the study and the specific anaiyses that led to the findings. In looking at our results, 
we expect that different people in the engineering community wiil have different reactions. 
Some findings confirm common beliefs, while others might contradict the reader’s own 
experiences or otherwise chaiienge expectations. Many of the findings have muitipie 
interpretations, and not all findings are directiy actionabie in the same way. A distinctive 
feature of this body of findings, besides the scope of the work, is that all of the results are 
grounded in data from a set of rigorously designed and conducted studies. To aid the reader, 
we have grouped the APS resuits by various aspects of the student experience. 

Persistence in Engineering and Comparison with Other Majors 

Persistence in engineering majors is comparabie to that in other majors; in other words, 
students who start in engineering majors tend to stick with their majors as much as students 
in other fieids. Even so, those who persist may have significant and important doubts about 
staying in their engineering majors. Those who ieave engineering majors are disproportion- 
ateiyfrom groups underrepresented in engineering, including first-generation coiiege 
attendees. This resuits in a iess diverse graduating ciass. In addition, few students migrate 
into engineering majors after starting coliege, resuiting in a net loss of students of more than 
15% (greater than most other majors). This iow in-migration is partiy reiated to the curricuiar 
inflexibiiity and overloaded nature of some program structures. Students who do not begin 
coliege as engineering majors need to take key prerequisites, which often necessitates 
extending their undergraduate studies by one or more terms. Noteworthy, however, is that 
some 10% of engineering graduates do migrate into engineering, and this group has strong 
representation of underrepresented groups (and therefore can contribute to diversifying 
engineering). 

We also see that there are multiple pathways into engineering, and supporting iess-traveled 
pathways has the potential for broadening participation in engineering. Students should be 
encouraged to explore and choose pathways through early-college experiences that are tied 
to key motivationai factors and that iet students “try engineering out.” Students can (and do) 
iearn about engineering through muitipie sources— e.g., reiationships with facuity, advisors, 
and peers; coursework; co-op/internship experiences; and extracurricuiar activities. 

Motivation 

Students are motivated to study engineering by a variety of factors, such as psychological/ 
personal reasons, a desire to contribute to the social good, financial security, or, in some 
cases, seeing engineering as a stepping stone to another profession. Some factors are 
strong among ali engineering students— for exampie, intrinsic psychoiogical and behaviorai 
motivation. Some factors have more influence with one demographic group than another. 

For example, being motivated by mentors is stronger among women, whereas being 
motivated by the “making" and “doing” aspects of engineering (behaviorai motivation) is 
stronger among men. 
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Enabling Engineering Student Success 


Motivation is related to several important outcomes. For first-year students, enjoyment of 
engineering for its own sake (psychological motivation) is correlated with intention to 
complete an engineering major, and, for seniors, it predicts intention to enter into 
engineering work or graduate school. Given these relationships, it is important for everyone 
responsible for engineering education to better understand the nature of student motivation 
and how it might be leveraged to attract a wide variety of students to engineering and to 
provide them with opportunities to explore different aspects of engineering. 

The Many Ways That Engineering Students Experience College 

Just as motivation to study engineering is not identical for all students, neither is the way 
students construct and experience their college education— /.e., how they combine 
coursework and extracurricular Involvement; how they engage in co-op, internship, and 
research opportunities; and how they make decisions about their future. Some students 
desire significant engagement in everything they do, others are more selective in their 
patterns of involvement, and some seem largely uninvolved in out-of-classroom activities. 
Even students who follow similar academic paths may experience their education differently. 
For example, students differentially experience curricular overload or pressure to represent 
their demographic group. Some of these trends are related to gender or underrepresented 
racial/ethnic minority (URM) status, whereas others are more aligned with underlying 
motivation and confidence factors. Still others may be Influenced by programmatic 
structures and institutional settings. These findings suggest opportunities for improved 
advising and curricular program design, based on a deeper understanding of what students 
desire from their college education and the many ways they go about constructing and 
experiencing this education. 

Learning about Engineering, Becoming an Engineer 

Students develop an engineering identity and learn about engineering from a variety of 
sources: co-op and internship experiences, their coursework and instructors, extracurricular 
activities, and personal contacts. We observed that these sources vary little by gender or 
URM status. On the one hand, APS findings showed that students were learning about 
engineering: By their senior year, most engineering students saw problem solving, 
communications, teamwork, and engineering analysis as key engineering competences and 
were using more engineering-specific language to express technical ideas. However, 
comparing juniors and seniors to first-years and sophomores, we saw that the more 
advanced students did not exhibit greater attentiveness to the broad context of engineering 
design problems (though women considered broad context more so than men on some 
engineering design tasks). In addition, seniors did not perceive professional and 
Interpersonal skills (e.g., leadership, teamwork, communication, and business ability) as 
being any more important than did their first-year counterparts, even having had project- 
based learning, design experiences, and, possibly, co-op or Internship experiences. These 
findings suggest that the typical engineering curriculum may not be doing enough to help 
students carry what they learn In first- and second-year math and science courses Into the 
more engineering-focused classes In their latter years. 

These gaps suggest that some students fall to Integrate the knowledge they are gaining 
about engineering from the various sources and across their years of study into a more 
complex, complete understanding of what it means to be an engineer. Furthermore, 
students do not always successfully transfer specific course knowledge and skills to real- 
world problems and settings. For instance, they may not anticipate how the teamwork skills 
they develop in courses using project-based learning are applied when working as an intern 
on a globally distributed design team. Alternatively, they may not recognize that the 
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organizational skills needed to manage multiple projects in their co-op assignment are 
similar in nature to the skills they learned in leading a student organization. 

Developing the Whole Learner 

Engineering students report experiencing considerable intellectual growth during their 
undergraduate years; they learn to apply key math and science support tools, and learn to 
take on substantial challenges in their design work. In addition, their college studies 
promote gains in confidence in many of the professional and practical skills increasingly 
called for in practice. However, studying engineering may mean students are not able to 
take advantage of other parts of a college education. For example, engineering students 
report lower gains in personal growth and fewer opportunities to study abroad than students 
in other majors. Some engineering students also report a sense of curricular overload. 

In addition, when compared with first-year students, seniors are less involved in engineering 
courses, are less satisfied with their instructors (though they interact with them more 
frequently) and are less satisfied overall with their college experience. In spite of these 
relative differences, seniors reported having significant learning experiences, especially 
those that were in-depth and presented them with a challenge. 

Positioning for Professional Success: Student Plans 

About 30% of the engineering students we studied had post-graduation plans focused 
exclusively on engineering (work and/or graduate school). These students were strongly 
motivated to study engineering for intrinsic psychological reasons and were likely to have had 
co-op and/or internship experiences. In general, these same students were among those 
who were less confident in their professional and interpersonal skills than those considering 
non-engineering professional endeavors post-graduation. 

Most other students conceived of their careers as combining engineering and non- 
engineering components. Some of these students expected different degrees of engineering 
specificity in their work, changing as their careers progressed. Others may still have been 
uncertain, even as graduation approached, as to whether an engineering or non-engineering 
path would be the best fit for them. These patterns might also have been influenced by the 
focus of the institution that students attended. In any case, faculty, staff, and programmatic 
structures generally do little to acknowledge (much less support and advise) students looking 
at combining engineering and non-engineering endeavors in their career plans. 

Early Experiences in the Work World 

Those students who enter the work world after graduating face challenges on multiple fronts. 
They find that the problems they are solving are more complex and ambiguous than the 
problems they solved in school. The structures of their new work environments are 
unfamiliar and multi-faceted, and it can be difficult for newly hired engineers to find the 
information they need. 

Sometimes, recently hired graduates feel that they are not allowed sufficient exposure to the 
“big picture" of where they and their work activities fit into the goals of the work group or 
company. These new hires also find that they are working with larger, more diverse teams 
than they experienced in school— teams that are composed of engineers and non-engineers, 
coworkers, and customers or clients. They must often learn new terminology and new 
communication skills. 
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Beyond the Academic Pathways Study 

We hope the new insights about engineering student pathways gained through APS research, 
coupled with practice-related questions provided in Subsection 2.10, will facilitate reflection, 
stimulate discussion, and eventually Inform action on campuses across the country. The APS 
team has already engaged multiple communities in productive discussions facilitated in 
multiple formats at a variety of major conferences. Better understanding the diversity in the 
experiences of our students will Inform how we design, deliver, and improve engineering 
education. This diversity in student experiences brings to mind questions about how 
teachers accommodate such a wide range of student goals, choices, and pathways. As 
discussed next, another component of CAEE’s research addressed aspects of the teaching of 
engineering. 

Investigating Faculty Approaches to Teaching: 

Studies of Engineering Educator Decisions (SEED) 

In the Studies of Engineering Educator Decisions work, we sought to gain insight into 
engineering teaching using an innovative approach: the collection and analysis of narratives 
about teaching decisions. To do this, we interviewed 31 engineering educators about two 
decisions that they had made: a planning decision (a decision made in advance of teaching) 
and an interactive decision (a decision made in the moment). Our interview protocol was 
designed based on principles from the Critical Decision Method, an approach used to study 
decision making in other domains. We then used the resulting narratives about the 
decisions and how they were made to investigate a variety of questions related to 
engineering teaching. 

• In analyzing the decision narratives to better understand educator decision-making, 
we found that most participants reacted positively to the emphasis on decisions and 
decision making, and that all were able to provide rationale for their decisions (with 
both time and allusions to prior decisions as common features of their rationale). We 
also learned that the participants collectively mentioned a variety of sources of 
information as being useful In decision making (although research was infrequently 
mentioned as a source), and we Identified five patterns in terms of satisfaction with 
their teaching decisions. 

• Looking beyond their decision processes and toward what additional information the 
decision narratives could reveal, we analyzed the narratives to explore educators’ use 
of teaching practices that are considered effective at helping students develop 
intrinsic motivation to learn. In this analysis, we found that engineering educators 
reported using a variety of teaching practices that are known to increase student 
motivation to learn, such as helping students see the relevance of material, helping 
students feel connected to the learning group, and helping students experience 
productive levels of engagement and challenge. We found less frequent mention of 
providing students with opportunities for autonomy, enabling all students to feel 
respected, and providing students with opportunities to demonstrate their growing 
competence. 

• Driven by the broad issue of how engineering educators conceptualize engineering 
students, we analyzed the decision narratives to learn more about how engineering 
educators differentiate among students. In this analysis, we found that all of the 
educators differentiated among students at some point, that student behaviors were 
the most prevalent basis for differentiating among students, and that differentiation 
based on other dimensions (e.g., what students know, their educational 
classifications, their social classifications) was also prevalent but less so. 
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• In addition to these analyses, we also investigated the benefits of engaging in 

research on teaching decisions. We observed that engaging in research on teaching 
decisions has professional development benefits for the researchers who analyzed 
the decision narratives, the researchers who collected the narratives, and even the 
educators who were asked to provide the narratives. 

In looking ahead, we believe the outcomes of this research may be useful for faculty 
development personnel in helping them to better understand their faculty clients. 
Additionally, the decision narratives themselves can be used by faculty developers to initiate 
fruitful discussions with faculty on problematic teaching issues. 

We also believe that these results represent a starting point for additional research. 
Variations of this work could focus on collecting decision narratives related to specific 
constraints (e.g., decisions about assessment, decisions about student projects, decisions in 
working with freshman). In Section 3 of this report, we also offer a variety of more specific 
research questions that could be explored. 

Finally, there are opportunities to bring these ideas together— for projects featuring not only 
the collection and analysis of decision narratives in specific domains but also active efforts 
to leverage (and document) the collection and analysis activities as professional 
development. 

Supporting the Development of Future Faculty: 

Engineering Teaching Portfolio Program (ETPP) 

The Engineering Teaching Portfolio Program was designed to assist engineering graduate 
students with an interest in teaching by advancing their thinking about teaching through the 
development and peer-based discussion of teaching portfolios. Each student’s portfolio 
consisted of a teaching philosophy statement, a diversity statement, and several annotated 
teaching artifacts. 

Significant ETPP outcomes include a comprehensive set of curricular and supplemental 
materials that are available for others to use, approximately 100 program “graduates,” and 
several small-scale spinoff efforts. In addition, the program’s research component informed 
the development of the curricular materials and can be used by others interested in 
supporting graduate students. 

ETPP is notable in the way it embeds opportunities to learn about teaching in the production 
of something inherently desirable to future faculty (the portfolio), includes conversations 
about diversity in prominent ways, and involves a way of talking about teaching that supports 
participation by people with a wide range of prior experiences. 

Our experiences with multiple offerings of ETPP suggest that the educational power of 
portfolio construction comes from consideration of the significant questions that can be 
associated with portfolio construction (e.g., who am I talking to, what exactly do I want to say 
about my teaching, who Judges teaching, how do I provide evidence of my strengths as a 
teacher, what counts as “good” teaching). Potential future work with ETPP includes more 
extensive data analysis to better understand how participants benefit from program 
participation, as well as additional program offerings and integration of APS results into the 
curriculum and supplemental materials. 
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Building Engineering Education Research Capacity and Community: 

Institute for Scholarship on Engineering Education (ISEE) 

The Institute for Scholarship on Engineering Education (ISEE) sought to cultivate a diverse 
community of engineering education researchers. In addition, the team formulated 
principles and models for advancing this community of schoiars. 

Three cycies of the Institute for Scholarship on Engineering Education were held. The three 
cycles involved a total of 49 engineering education researchers (Institute “Schoiars”) 
representing 20 institutions. Twenty (40%) were women and 17 (36%) were 
underrepresented minorities. Ail academic ranks (and other roies) were represented: 6 fuli 
professors; 12 associate professors; 9 assistant professors; 13 graduate students; and 9 
staff members, including administrators and advisors. 

Each ISEE cycle consisted of five main phases: (1) designing and/or adapting the Institute 
modei, (2) recruiting Scholars for the current year’s Institute, (3) a week-iong Summer 
Summit kick-off event at the host school, (4) activities during the academic year to support 
Schoiars in conducting their studies, and (5) a culminating Leadership Summit event. The 
Summer Summit was designed to engage the Scholars in the process of engineering 
education research, introducing many to new techniques and ideas in educational research. 
Activities and discussions during the Summit helped Scholars refine their research 
questions, decide on appropriate methodology, and, very importantly, to form a community 
that couid be sustained beyond the week-iong meeting. After the Summit, Scholars typically 
returned to their respective campuses to conduct their research, with frequent eiectronic 
communication and interaction with feilow Scholars and the ISEE team. 

Each Institute had a different theme. The individual projects for the 2004-2005 Institute 
primarily focused on classroom changes under the broad theme of “classroom as lab.” For 
the 2005-2006 Institute, Scholars worked on projects aimed at impact on engineering 
education at their campus (/.e., a theme of “campus as lab”). The 2006-2007 theme was 
“Advancing Engineering Education Research to Meet the Needs of the 21st Century.” 
Schoiars were recruited through a competitive, nationai appiication process and were asked 
expiicitly to consider issues of diversity in their projects. As such, the focus of the 2006- 
2007 Institute was “nation as iab.” 

The Institutes had a powerful impact on the participating Scholars in three broad areas: 
building skilis, knowledge, and experience; broadening their career paths; and helping to 
foster their membership in the broader community of engineering education researchers. In 
terms of impact on their institutions. Scholars’ projects addressed existing concerns on 
several campuses. Examples include examining a set of engineering fundamentais courses 
on one campus, investigating the benefits of “empowering” students at another campus, and 
supporting Hispanic students transferring from community colieges. 

ISEE served as a model for others interested in organizing simiiar community-building 
activities. A paper describing the design of the Institutes and an example schedule for the 
week-long kick-off event are available on the CAFE web site. 

Finally, a research study of 13 engineering education researchers detailed two significant 
aspects of their pathways into the fieid of engineering education research: the importance of 
a community of practice perspective and the development of composite identity. This study 
further extended our understanding of capacity buiiding for engineering education research. 
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Getting the Word Out: 

Publications, Presentations, Research and Program Resources, and People 

The CAEE team recognized from the outset that a significant part of our mission was to get 
the word out about the work of the Center. A fundamental part of this activity was sharing 
news about the research and results with as wide a variety of audiences and in as many 
different venues as possible. These dissemination activities began early In the life of the 
Center and are continuing after the formal end of the grant period. 

From January 2003 to June 2010, CAEE productivity included 

• over 130 papers and journal articles In both engineering education and education 
publications; 

• 9 plenary, keynote, and invited presentations at national conferences and meetings; 

• 9 conference special sessions; and 

• more than 25 workshops to a wide variety of audiences. 

In addition to published research findings and presentations, CAEE created a set of materials 
that can be used by others In conducting their own research. These materials include two 
surveys, four sets of Interview protocols, two engineering design task exercises, and program 
design and materials for ETPP and ISEE. The Academic Pathways Study team prepared a 
report and complete documentation package describing the design and implementation of 
the APS. We have already seen that these APS tools and materials are being used 
extensively by other researchers. 

Capacity building was also a critical part of the Center’s impact. An explicit goal in assem- 
bling the team was to combine researchers from both engineering and education depart- 
ments who had a mix of quantitative and qualitative research expertise. Over the course of 
the grant, the Center grew to involve 63 faculty members and staff, 41 graduate students, 
and almost 50 undergraduates during the period 2003-2010. As research scientists and 
graduate students moved on to faculty positions at other campuses, they typically continued 
their involvement with CAEE, spreading the Center’s influence even further. 

Future Work 

The Academic Pathways Study results call attention to certain areas of educational research 
that warrant further analysis. For example, considering the diverse programs and student 
perspectives we observed across the institutions we studied, it makes sense that other 
institutions have their own nuances to be explored. We also observed substantial changes in 
students between their first and senior years, both in terms of learning and development. 
This warrants further inquiry into the middle years— the experiences of sophomore and Junior 
level students. Further, APS longitudinal research focused on the experiences of students 
who spend the entirety of their undergraduate careers at one institution. However, APS 
cross-sectional research shows that this represents only a portion of engineering students. 
Studies of community college and transfer students are becoming more Important as 
students increasingly follow this academic path. Finally, there are Important questions 
concerning students who never consider or enter engineering. 

Given that the current Studies of Engineering Educator Decisions work focused on 
participants from a single Institution, extending the research to confirm or refine findings 
reported here would be valuable. Such additional data could also be used to further 
Investigate issues such as the role of research results In informing teaching decisions; the 
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relationship between satisfaction, dissatisfaction, and additional change; and the broad 
issue of how engineering educators conceptualize students. 

Building on the Engineering Teaching Portfolio Program efforts, we could further explore the 
ways in which the portfolio construction activities help participants reflect on their existing 
ideas about teaching and ultimately develop a more sophisticated, integrated, personalized, 
and actionable understanding of teaching. Building on research on the community of 
engineering education scholars that was conducted as part of the Institutes for Scholarship 
in Engineering Education, we could expand our investigations into how people enter and 
navigate the fieid of engineering research, even as the community itself is growing. 

The future work ideas above represent direct extensions of our work. In our report, we also 
go farther by offering a broad set of research questions organized into seven areas: 
questions related to (1) student pathways; (2) student learning of engineering; and (3) the 
role of significant learning experiences; as weil as questions related to (4) engineering 
knowing, (5) teaching engineering, (6) researching issues in engineering education, and (7) 
bringing about change in engineering education. 

Closing Comments 

Engineering education is a rich and vibrant area for research, with many opportunities for in- 
depth scholarship that can contribute significantly to improving engineering education for 
many constituencies. In 2010, as CAEE comes to an end, the engineering education 
community is much larger, more distributed, more interdisciplinary, and has expertise in a 
wider range of research methods than when we began our work in 2003. 

CAEE contributed to this expanding fieid during our seven years of funding, not oniy through 
the generation of a rich body of research results, but also by demonstrating the scope of 
research and program activity that a large center is uniquely capable of accomplishing. CAEE 
also contributed to the growth of the engineering education community through the many 
individuals who were involved directly and collaboratively with the Center and through the 
community-buiiding efforts of ISEE. 

We feei that CAEE has been a significant contributor to growth in these areas, as weil as to 
important efforts of heiping to use research findings to improve engineering education. As 
the engineering education community moves forward, we anticipate further research-based 
improvements to engineering education, ensuring that a diverse cadre of engineering 
graduates are prepared for the chalienges they wiil face in the coming years. 
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1 Introduction 


1.1 Situating Our Work 

Today’s engineering graduates wiil soive tomorrow’s problems in a world that is advancing 
faster and facing more criticai challenges than ever before. This situation creates significant 
demand for engineering education to evolve in order to effectiveiy prepare a diverse 
community of engineers for these chaiienges. 

Such concerns have led to the publication of visionary reports that heip orient the work of 
those committed to the success of engineering education. For exampie, the Engineer of 
2020 (Nationai Academy of Engineering 2004) offers a biueprint for the knowledge and skills 
future engineers will need in order function effectively in the future. Creating a Cuiture for 
Schoiariy and Systematic innovation in Engineering Education (Jamieson and Lohmann 
2009) emphasizes the need for a culture of innovation in engineering education in order to 
create and disseminate educational activities and curricuia that are effective at preparing 
students for the future. Engineering for a Changing Worid (Duderstadt 2008) presents a 
systems perspective on innovation, locating innovation not just within the curriculum but 
within a larger framework that is composed of engineering education and engineering 
practice. 

Research in engineering education is central to aii of the visions described in these reports. 
Research can shed light on how students develop their competencies, the effectiveness of 
innovations, and how a systems perspective affects the endeavor. The report Educating 
Engineers: Designing for the Future of the Fieid (Sheppard et ai. 2008) represents a 
significant contribution to this research space, with its goal to “understand, through field 
research, how the educational practices of the schools form future engineers.” (p. x/x). A 
broad sense of the research needed for engineering education was addressed by the NSF- 
sponsored conversations referred to as the Colioquies on Engineering Education (Adams et 
ai. 2006). 

Research on the student experience is a fundamentai kind of 
research for informing the evoiution of engineering education 
A broad understanding of the engineering student experience 
involves thinking about pathways, navigation, and decision 
points— how students choose engineering programs, navigate 
through their programs, and then move on to Jobs and 
careers. Further, looking at students’ experiences broadly 
entails not Just thinking about their learning (/.e., skill and 
knowledge development in both technical and professional 
areas) but also their motivation, their identification with 
engineering, their confidence, and their choices after 
graduation. 

In actuality, there is not one singular student experience, but rather many experiences. 
Research on the engineering student experience can iook into systematic differences across 
gender, disciplines, and campuses; gain insight into the experiences of underrepresented 
students; and create a rich portrait of the changes from students’ first year through 
graduation. Such a broad understanding of the engineering student experience can serve as 


Research on the student 
experience is a fundamental 
kind of research for enabling 
innovation in engineering 
education. 
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inspiration for designing innovative curricular experiences that support the many and varied 
pathways that students take on their way to becoming an engineer. 

However, an understanding of the engineering student experience is clearly not enough to 
create innovation. For example, we need educators who can use the research in the context 
of their own teaching and broader teaching innovations. We need educators who are 
prepared to innovate, which in turn requires being prepared to use the research on the 
student experience. This involves not only preparing tomorrow’s educators with conceptions 
of teaching that enable such innovation but also understanding how today’s educators make 
teaching decisions. 

We also need to be concerned about creating the capacity to do such research— we need 
more researchers. While this can involve dedicated programs to train Ph.D. level 
researchers, we need additional innovative ways to create capacity. One promising approach 
is to work with educators who are interested in engaging in research, supporting them as 
they negotiate the space between their current activities and their new work in engineering 
education research. To fully support this process, we must also investigate what is required 
for educators to engage in such a path. 

This report describes the work of the Center for the Advancement of Engineering Education 
(CAEE)— work that addresses the issues highlighted above. CAEE engaged in four threads of 
activity: 

• Academic Pathways Study (APS, 2003-2010) 

• Studies of Engineering Educator Decisions (SEED, 2006-2010) 

• Engineering Teaching Portfolio Program (ETPP, 2003-2006) 

• Institute for Scholarship on Engineering Education (ISEE, 2003-2008) 

These activities all involved an emphasis on the people in the engineering education system: 
students, educators, and researchers. The Center activities involved concurrent and 
interwoven emphasis on both research and capacity building. For example, while the first 
two efforts (APS and SEED) focused on research and the second two (ETPP and ISEE) were 
primarily capacity-building efforts, significant capacity-building outcomes were part of the 
first two efforts, and the last two efforts addressed important research questions. Finally, 
our activities were not only motivated by a desire to support future innovation, but 
themselves involved innovation— from the scale of the Academic Pathways Study, to the 
novelty of the Studies of Engineering Educators Decisions, to the emphasis on diversity in the 
Engineering Teaching Portfolio Program, and the flexibility of the Institute model. In the rest 
of this report, we present the results of this work. 
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1.2 A Short History of the Center 

1.2.1 The Center 

The Center for the Advancement of Engineering Education (CAEE) began research in January 
2003 as one of two national higher-education Centers for Teaching and Learning funded by 
the National Science Foundation that year. Two divisions of the NSF provided support: 
Engineering Education and Centers (Engineering Directorate) and the Division of 
Undergraduate Education (Education & Human Resources Directorate). Originally funded for 
2003-2007, supplementary funds from the Engineering Directorate allowed additional 
analysis and dissemination to continue through 2010. 

1.2.2 The Team 

CAEE began as a team of scholars from Colorado School of 
Mines, Howard University, Stanford University, the University of 
Minnesota, and the University of Washington (the lead 
Institution). During the course of the grant, the team grew to 
include researchers from other institutions including Purdue 
University, Olin College of Engineering, Virginia Tech, University 
of Illinois at Champaign-Urbana, and the University of 
Rochester. Over the duration of the grant, team members 
included 63 faculty, research scientists, and staff; 41 graduate 
research assistants; and almost 50 undergraduates who were 
Involved in the research. 


1.2.3 The Research 

CAEE research was focused on the broad areas of 

• the engineering undergraduate learning experience and 
school-to-work transition, 

• understanding engineering educator teaching decisions, 

• the professional development of engineering graduate 
students using teaching portfolios, and 

• activities and models for expanding the engineering 
education research community. 

These areas aligned with four major strands of the research 
which are described below: 

The Academic Pathways Study (APS, 2003-2010), led by Sheri Sheppard, represented the 
most In-depth portion of CAEE’s research with approximately 80% of the personnel 
resources. APS activities included the longitudinal (160 participants) and cross-sectional 
(over 4,200 participants) studies of engineering undergraduates’ learning experiences. In 
addition, over 100 early career engineers and several of their managers participated in 
investigations of the transition to work. The APS also included smaller-scale, targeted 
studies that examined specific aspects of engineering student learning and experiences. 

The Studies of Engineering Educator Decisions (SEED, 2006-2010), led by Jennifer Turns, 
investigated the teaching decisions of 31 engineering faculty using a semi-structured 


Over 5, 700 individuals (including 
undergraduate students, 
graduate students, early-career 
engineers, practicing educators, 
faculty, and administrators) 
participated in CAEE activities. 


The CAEE team included over 
100 faculty members, research 
scientists, graduate students, 
and staff, as well as almost 50 
undergraduate students. 
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interview protocol. Participants represented nine engineering departments and a range of 
academic ranks from non-tenure track to assistant, associate and full professors (with 
several also serving in administrative positions). The sample included nine women. 

The Engineering Teaching Portfolio Program (ETPP, 2003-2006), led by Jennifer Turns and 
Angela Linse, was designed to use teaching portfolios to enhance the professional 
development of engineering graduate students. In iterative development of the ETPP, the 
team used a semi-structured interview protocol and field observations with over 100 
participants. The curriculum and supplemental materials were outcomes of this study, in 
addition to the research findings. 

The Institute for Scholarship on Engineering Education (ISEE, 2003-2008), led by Robin 
Adams, conducted three year-long Institutes that were specifically designed to expand the 
engineering education research community (involving 49 researchers representing 20 
institutions). The design and implementation of the three Institutes served as a basis for 
developing models in engineering education research that can be adopted and adapted by 
others. Thirteen Institute Scholars also participated in a companion study of the pathways of 
scholars into engineering education research. 


1.3 Goals of This Report and a Reader's Guide 

1.3.1 Goals of This Report 

This report provides an overview of the work of the CAEE team during the seven years of the 
grant. The report accomplishes several goals: 

• summarizes highlights from the findings, organized by the major themes of 

o the engineering student learning experience, 
o faculty approaches to teaching, 

o development of graduate students interested in teaching careers, and 

o building the community of engineering educators and engineering education 
researchers 

• includes references to publications for further investigation by the reader 

• lists key CAEE-developed materials for use by other researchers 

• offers two sets of questions— one that can serve as a guide for using our results 
locally, and a second set that discusses future research directions 

• serves as encouragement and a model for those undertaking similar work 

1.3.2 Reader’s Guide to the Sections 

Section 1 provides a general framing for our research and provides a brief overview of CAEE 
including the major research areas. 

Section 2 presents findings on undergraduates and early career engineers from the 
Academic Pathways Study (APS). This extensive section concludes with a summary of 
findings (Subsection 2.10) that includes a set of research-based “local inquiry questions” 
(also compiled in Appendix D) that can guide efforts to improve the undergraduate 
engineering experience. 
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Section 3 presents findings from the research into engineering teaching (SEED, Studies of 
Engineering Educator Decisions). 

Section 4 describes the deveiopment of an innovative portfolio program and accompanying 
tools to support graduate students interested in teaching careers (ETPP, the Engineering 
Teaching Portfolio Program). 

Section 5 gives details and examples of community buiiding in engineering education drawn 
from the Institute for Scholarship on Engineering Education (ISEE) program and the three 
year-long Institutes. 

Section 6 summarizes CAEE activities to get the word out to a variety of audiences at both 
local and national levels. It also provides a list of resources that were developed by the team 
and can be used by others. (These resources are available through the CAEE web site and 
include survey and interview instruments and a “behind the scenes” look at the design and 
development of the Academic Pathways Study.) 

Section 7 presents ideas for further work to effect change on campuses and conduct more 
research. 

In addition, the report contains five appendices: 

• Appendix A: References and Cumulative Bibliography 

• Appendix B: Cumulative Team List and Advisory Board Members (2003-2010) 

• Appendix C: APS Headlines (summarizing Section 2) 

• Appendix D: Local Inquiry Questions (drawn from Section 2.10) 

• Appendix E: Looking Ahead: Ideas for Future Research (expanded from Section 7) 
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2 Student Learning Experiences: 
The Academic Pathways Study 


This section of the report discusses CAEE’s research on the educationai experiences of 
engineering undergraduates as examined in the Academic Pathways Study (APS). Discussion 
of findings is organized by the iarge themes that emerged during the course of anaiysis. in 
some cases, findings may not be especiaiiy surprising, but they do provide empiricai 
confirmation and form part of the iarger story, in other cases, intriguing findings bring up 
questions that merit further research. We hope that the range of findings we present wiii 
interest engineering educators, poiicy makers, and other engineering education researchers. 
The seiected findings create a rich portrait of the iearning experience of engineering 
undergraduates as they move through four years of schooi and beyond. 

After an overview of the APS’s overarching research questions and methods, most of 
Section 2 describes a broad range of resuits, organized by theme. These resuits draw on 
over 100 papers based on the APS research, three dissertations, severai unpubiished 
anaiyses, and seiected works by other researchers. The ciosing subsection contains broad- 
brush summaries of the key APS research findings that are described in greater detaii in the 
preceding subsections, it aiso contains questions informed by this research for facuity, 
administrators, and staff to use in considering how to better support student success in 
foiiowing an engineering pathway through their programs. The topics covered in this section 
are as foiiows: 

• 2.1 Overview of the APS: Research questions, sampies and cohorts, methods 

• 2.2 The Coiiege Experience: Engineering students compared to other majors 

• 2.3 Motivation to Study Engineering: Motivationai factors in choosing to study and 
persist in engineering 

• 2.4 The Engineering Coiiege Experience: Educationai experiences as reiated to 
demographics, confidence, motivation, and other factors 

• 2.5 Engineering Knowiedge, Conceptions, and Confidence: Understanding of and 
confidence in engineering, and engineering practice 

• 2.6 Engineering Design Knowiedge, Conceptions, and Confidence: Understanding of 
and confidence in engineering design 

• 2.7 Looking Beyond Graduation: Student Pians: Post-graduation pians of engineering 
students 

• 2.8 Looking Beyond Graduation: Experiences in the Work Worid: Eariy-career 
experiences in the engineering workpiace 

• 2.9 Summarizing Resuits about Diversity: Findings reiated to underrepresented 
students 

• 2.10 Enabiing Success for Engineering Students: Summary of the findings and 
questions to ask in guiding efforts to enabie student success 


18 


Enabling Engineering Student Success 


2.1 Overview of the APS 

The primary goal of the Academic Pathways Study was to create a rich and wide-ranging 
portrait of the undergraduate engineering learning experience, using a variety of research 
methods and relying on the students’ own words for much of the data. 

The APS represents the largest portion of CAEE’s research, with approximately 80% of the 
center’s budget allocated to researcher support. During the course of the APS, over 130 
faculty, research scientists, graduate and undergraduate research assistants, and staff 
representing 12 universities and six national organizations were involved In the research. 
Detailed research design began in early 2003, and data were collected during the 2003-04 
through 2007 -08 academic years. The original funding was from 2003 to 2007, and NSF 
provided supplemental funds to enable two additional years of work. Data analyses 
continued Into 2010. 

The research questions being addressed by APS are listed in Table 2.1-A and consider how 
today’s educational systems support students learning to be engineers. 


Table 2.1-A: APS research questions 


Focus 

Research question 

Skills 

How do students’ engineering design skills and understanding of engineering 
practice develop and/or change over time? 

Identity 

How do students come to identify themselves as engineers? How do these 
identities change as they navigate their education? 

Education 

What elements of students’ engineering educations contribute to changes 
examined in the skills and identity questions above? 

Workplace 

How do students conceive of their careers? What skills do early-career engineers 
need as they enter the workplace? 


2.1.1 Individual Studies 

To investigate these research questions, we designed a series of longitudinal and cross- 
sectional studies of engineering undergraduates’ learning experiences and transition to 
work. In addition, we drew on the National Survey of Student Engagement (NSSE) data set to 
provide the basis for a large-scale comparison between engineering students and students 
from other academic disciplines. 

Table 2.1-B summarizes the number of participants and institutions/organizations for the 
different studies. Following the table, the studies are described briefly, including goals, 
methodology, and complete duration (including initial study design, data collection, 
subsequent analyses, and dissemination). For a more detailed description of the APS design 
and methodology, see the CAEE technical report “An Overview of the Academic Pathways 
Study: Research Processes and Procedures” (Sheppard et al. 2009; available on the CAEE 
web site). 
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Table 2.1-B: APS cohorts, samples, and studies with number of participants, 
institutions/organizations, and methods (S: survey, I: structured and/or semi- 
structured (ethnographic) interview, F: focus group, 0: observation, D: engineering 
design task) 


Study 

Participants 

Institutions/ 

organizations 

Data 

collection 

Methods 

Longitudinal Cohort 

160 

4 

2003-2007 

□□□□ 

Broader Core Sample 

842 

4 

2007 

□ 

Broader National 
Sample (APPLES) 

4,266 

21 

2008 

□ 

NSSE Comparative, 
Longitudinal Data Set 

11,812 

247 

2002-2007 

□ 

Single-School, 

Cross-sectional 

Sample 

160 

1 

2005-2006 

BO 

Transition to 
Workplace Studies 

101 new hires, 
15 managers 

14 

2007-2008 

B 

□ 

Difficult Concepts 
Study 

19 students, 
23 faculty 

1 

2004-2006 

BO 


Longitudinal Cohort (2003-2009) 

The Longitudinal Cohort consisted of 160 undergraduate engineering students (40 at each of 
four diverse campuses). The students were paid a stipend to participate in the study from 
2003 to 2007, beginning with their first year in college and into their fourth year. 
Oversampling increased the number of participants from underrepresented groups in 
engineering. The initial sample comprised approximately 61% men and 39% women. Just 
under 60% of the participants were white or Asian-American, with the rest being from 
underrepresented racial/ethnic minority (URM) groups. To put these numbers into 
perspective, at the national level, 19.5% of engineering graduates are female, and 12% are 
from underrepresented racial/ethnic groups (National Science Foundation 2010; see 2006 
data (most recent available) in Tables 4 and 6). 

The APS research team used four primary data collection methods for the Longitudinal 
Cohort: surveys, structured and semi-structured (ethnographic) interviews, observations, and 
short engineering design tasks, as described below. In addition, academic transcripts were 
collected for all participants, and exit interviews of those leaving an engineering major were 
conducted. 
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• The Persistence in Engineering (PiE) Survey was designed to identify and characterize 
the fundamentai factors that influence students’ intentions to pursue an engineering 
degree and to practice engineering as a profession. The survey was buiit foiiowing an 
extensive review of engineering education iiterature and previous nationai surveys on 
undergraduate education. The PiE survey was administered to the entire Longitudinai 
Cohort seven times: twice in each of the students’ first three years and once in their 
senior year. 

• The structured interviews extended and expanded on the PiE survey with a focus on 
specific information reiated to the students’ engineering education and identity 
deveiopment. The structured interviews used a fixed set of questions and iasted 
approximateiy one hour. A primary goai was to eiicit quaiitative data in the form of the 
students’ own words. Twenty-four students at each of the four campuses participated 
in the structured interviews through their first three years. 

• Semi-structured (ethnographic) interviews and ethnographic fieid observations were 
used to gather data from a smaiier set of participants (16 per campus for the semi- 
structured interviews, of which 8 aiso participated in iimited fieid observations in their 
first year). The focus was on extending the understanding of both the iocai cuitures of 
engineering student experiences and students’ pathways through these experiences. 
Researchers examined activities such as intense project work, exam periods, and 
extracurricuiar activities. 

• The engineering design tasks were short (approximateiy 10 minutes) “engineering 
tasks” designed to investigate how students approach open-ended engineering design 
probiems at various stages of their academic careers. Foiiowing each design task, 
participants were asked to discuss their answers as a means for researchers to better 
understand students’ approaches and the reasoning behind them. The design task 
data were suppiemented by responses from specific questions inciuded in the PiE 
survey that reiated to students’ conceptions of design and engineering work. 
Administration of the design tasks varied during the course of the study and was 
coupied with the structured and/or semi-structured interviews, depending on the year. 

Subsection 6.2 of this report describes avaiiabie resources, inciuding the surveys and 
protocois for each particuiar method mentioned above. 

Note that for the quaiitative portion of the research (interviews and design tasks), student 
participants were assigned pseudonyms. These pseudonyms are inciuded in many of the 
quotations that appear in the foiiowing sections. 

The four schoois that participated in the Longitudinai Cohort were from across the U.S. and 
of varying size and institutionai type. They were a pubiic research university devoted to 
engineering and appiied science; a comprehensive, historicaiiy Biack, private university; a 
private research university; and a very iarge, pubiic research university. 

For additionai information on the design of the Longitudinai Cohort research and the PiE 
survey in particuiar, see Eri§ et al. 2005. 

Broader Core Sample (2006-2008) 

The Broader Core Sampie served as a piiot for the Academic Pathways of Peopie Learning 
Engineering Survey (APPLES, described in the next paragraph), as weii as a means to gather 
further data from additionai students at the four Longitudinai Cohort schoois. The 
approximateiy 10-minute survey was given to a cross-sectionai group of over 800 first-year to 
senior undergraduates in eariy 2007. Most participants were engineering majors, but the 
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sample also Included “switchers,” or students who were interested In engineering at one 
time and subsequently switched to a non-engineering major (about 10% of the sample). 
(Switchers are discussed further in Subsection 2.2.2.) 

Broader National Sample: APPLES (2007-2009) 

The Broader National Sample, also known as the Academic Pathways of People Learning 
Engineering Survey or APPLES, was a cross-sectional survey study of over 4,200 engineering 
undergraduates at 21 campuses of varying size, location, student demographics, and 
mission. The survey was administered in late winter through early spring of 2008. The 
APPLE Survey is a shorter version of the Longitudinal Cohort PIE survey described above. 
Similar to the Broader Core Sample, the Broader National Sample included not only 
engineering students but pre-engineering and “switcher” students as well. The sample 
included approximately 16% underrepresented minorities (URM) in engineering. Women and 
men represented approximately 35% and 65% of the sample, respectively. (These 
race/ethnicity and gender data were obtained from a multiple-choice question on the APPLE 
Survey.) For more detailed descriptions of the Broader Core and Broader National Sample 
(APPLES) surveys, see Sheppard et al. 2010 and Donaldson, Chen, et al. (2007, 2008). 

NSSE Comparative, Longitudinal Data Set (2008-2009) 

We collaborated with researchers from the National Survey of Student Engagement (NSSE) in 
order to situate APS findings about engineering students in the broader context of other 
college students with a wide range of majors. The NSSE sample included a total of 11,812 
students at 247 institutions who responded to the NSSE survey as first-year students in the 
spring of 2002, 2003, or 2004 and again as prospective graduates three years later, in the 
spring of 2005, 2006, and 2007, respectively. (Lichtenstein et al., forthcoming; McCormick 
eta/., forthcoming) 

Single-School, Cross-sectional Sample (2005-2006) 

The Single-School, Cross-sectional Sample participants were drawn from a large, public 
university in the Midwest that was not involved in the original Longitudinal Cohort research. 
The sample targeted 160 undergraduate students distributed across their first through 
fourth years. The PIE survey was administered in the fall and spring of the 2005-06 
academic year. In addition, six focus groups were conducted in the spring of 2006 to 
supplement quantitative information from the survey and with an emphasis on including 
transfer students. (Korte and Smith 2007) 

Transition to the Workplace Studies (2006-2009) 

The Transition to the Workplace Studies focused primarily on the early career experiences of 
recently-hired graduates. The studies involved over 100 early career engineers (and 15 of 
their managers) employed in a range of private companies and public agencies. Data were 
collected using interviews and observations. Related findings concerning post-graduation 
plans are based on survey data collected from the Broader National Sample (APPLES). 

Six distinct data sets were collected: 

• Thirty newly hired graduates and six managers at a global vehicle manufacturing 
company 

• Eighteen newly hired graduates at a national food manufacturing company 

• Nineteen newly hired graduates and four managers at a smaller manufacturer of 
computer components 
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• Nineteen newly hired graduates and five managers at a state-government agency for 
transportation 

• Seven newly hired graduates: five at a county public works department, one at a state 
transportation agency, and one at a small aerospace firm 

• Eight engineering graduates (from one of the APS Longitudinal Cohort schools) 
employed at a variety of engineering firms 

Difficult Concepts Study (2004-2006) 

A separate study focused on difficult concepts in engineering mechanics (e.g., force) and 
electrical circuits (e.g., voltage). Nineteen seniors (10 majoring in electrical engineering and 
9 majoring in civil/mechanical engineering) at a public research university devoted to 
engineering and applied science participated in hour-long interviews designed to probe their 
understanding of these concepts. Prior to beginning the research with the students, 10 
faculty members in circuits/electrical engineering and 13 in mechanical/civil engineering 
were surveyed using an iterative Delphi process to develop the interview questions. 

2.1.2 Notes About Section 2 

The remainder of this section provides selected findings from APS research; we did not 
attempt to provide an exhaustive summary of all of the work accomplished during the study. 
In addition to characterizations of engineering students as a group, some of the selected 
findings are the results of comparisons between various subsamples of interest, e.g., 
comparisons by class standing, by gender, and by race/ethnicity. For those readers who are 
interested in delving deeper into the research, we have included references in the text that 
identify the sources for specific findings, as well as a full list of our publications to date in 
Appendix A, Subsection A.2. 

In the interest of focusing directly on the APS findings, we have included only a few 
references to other work from the literature, e.g., to provide useful theoretical background 
and/or terminology. This limited set of external references is listed in Appendix A, 

Subsection A.l. Those interested in seeing our discussion and interpretation of results 
situated in the context of the broader engineering education literature are directed to the 
individual APS papers and articles cited throughout this section. 

Data from the APPLE Survey instrument administered to the Broader National Sample are 
referred to throughout the report and are not referenced each time they are discussed. The 
quantitative APPLES (4,266 students) and PIE survey (160 students) data are accompanied 
by excerpts, often including quotations, from the qualitative research (“smaller-n” interviews 
and design tasks). The discussion of APPLES results focuses on comparisons of the first year 
and senior year data. See the full APPLES report for more details (Sheppard et al. 2010). 

Prior APS-related publications refer to the cohorts and samples in slightly different ways. The 
APS overview technical report (Sheppard et al. 2009) refers to the Single-School, Cross- 
Sectional Sample as the Cross-Sectional Cohort and the Transition to the Workplace Studies 
as the Workplace Cohort. Earlier publications refer to the Longitudinal Cohort as Cohort 1, 
the Broader Core Sample as Cohort 3, and the Broader National Sample as Cohort 4. The 
Transition to Workplace Studies grew out of research efforts initially referred to as Cohort 2. 
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2.2 The College Experience 


The Academic Pathways Study involved multiple cohorts/samples of students, the largest of 
which included 4,266 undergraduates from 21 institutions across the U.S who participated 
in the Academic Pathways of People Learning Engineering Survey (APPLES). As part of an 
extended analysis of the APPLES data, we also collaborated with researchers from the 
National Survey of Student Engagement (NSSE) to compare our data on engineering 
undergraduates with NSSE’s findings about students in engineering, as well as other majors. 
NSSE researchers performed a search of students who took the NSSE survey in both their 
first and senior years and found nearly 12,000 useable cases representing a broad range of 
undergraduate majors. This subsection presents details of our comparative analysis, 
accompanied by additional APS results relating to persistence in engineering majors. See 
Subsection 2.1.1 for details about APPLES and the NSSE data set. 

2.2.1 Engineering Students Compared with Other Coliege Students 

Other researchers (e.g., Astin 1993; Sax 2008; Beyer, Gillmore, and Fisher 2007; 

Fairweather 2009) have shown the value of looking at how the college experience is affected 
by a student’s disciplinary field. Their research approach and sometimes provocative 
findings inspired us to compare engineering students’ experiences to those of students in 
other fields. We considered engineering in comparison with six groups of majors: 
science/technology/math, computer science, business, social sciences, arts and humanities, 
and a group composed of all other majors. The engineering majors made up 5% of the 
seniors we considered, as shown in Figure 2.2-A. This percentage of engineering students is 
comparable with their representation in the U.S. college student population more generally 
(National Science Foundation 2010, Table 5-2006; Chen 2009). This subsection draws 
extensively on the analysis we did with the NSSE data set to offer a broad overview of the 
college experience and complements these findings with explanatory insights from our 
detailed Longitudinal Cohort study. 


Figure 2.2-A: The majors of graduating seniors in the NSSE data set (N = 11,817) 
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Figure 2.2-B: Persistence in matriculating major (left bars), migration into major 
(middle bars), and net loss/gain in major (right bars) in the NSSE sample 
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Engineering majors are as iikeiy to persist as are 
other majors. 


Definitions 


There is strong evidence that persistence among 
engineering students is comparable to other majors, 
as shown in Figure 2.2-B. However, engineering has 
the lowest percentage of migration in. This disparity 
results in engineering having the greatest percentage 
decrease in students relative to other majors when 
both persistence and in-migration are considered. In 
Figure 2.2-B, the rightmost bars show net change, 
where engineering has a 16% drop in students by the 
senior year. 


persistence: ratio of students 
remaining as seniors in a major 
area relative to the number 
matriculating in that major 

migration in: proportion of 
students who as seniors are in a 
major area different from the one 
in which they matriculated 
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There are similarities among, but also differences between, engineering majors and other 
majors with respect to learning and college-experience measures. 

Engineering students are similar to non-engineering majors in a variety of ways. For 
example, they are equally engaged with their studies (e.g., interaction with faculty, 
participation in extracurricular activities) as students in other majors. In addition, they are 
similar in other factors, ranging from self-reported GPA to overall satisfaction. However, 
engineering students differ from students in other majors in certain learning gains (Figure 
2.2-C). Engineering students make greater gains in practical competence and higher order 
thinking, but lower gains in personal and social development and general education. 
(McCormick et al., forthcoming; Lichtenstein et al. 2010) 


Figure 2.2-C: Mean values for selected engagement variables for seniors in the 
NSSE sample, by major group 
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Figure 2.2-D: Percentage of seniors in the NSSE sample who have already done or 
plan to do each enriching experience, by major group. “Culminating senior 
experience” includes capstone courses, theses, projects, and comprehensive exams. 
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In terms of enriching educational experiences, engineering majors are more likely to have a 
culminating senior experience (e.g., a capstone project) but less likely than other majors to 
study abroad or take foreign language coursework (Figure 2.2-D). Additionally, engineering 
majors report spending more time preparing for their courses than students in any other 
major. (Lichtenstein etal. 2010, Stevens etal. 2007) 
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2.2.2 Comparing the Demographics and Pathways of Switchers and in-Migrators 


Definition 

switcher (or non-persister): a 
student who began college in 
engineering and has switched 
into a different major area {e.g., 
the sciences, arts & humanities) 
by senior year 


Engineering persisters are more iikeiy to be maie and white, and iess iikeiy to be first- 
generation coilege students. 

The demographic picture of engineering persisters 
compared to switchers that is painted by the NSSE 
data set provides insights about the extent to which 
engineering draws in and retains a diverse set of 
students. (Our definition of switcher (right) buiids on 
the STEM education work of Seymour and Hewitt 
(1997) but focuses specificaiiy on engineering.) The 
NSSE data indicate that the engineering persister 
group has proportionaily fewer women and 
underrepresented racial/ethnic minority (URM) 
students in it than the engineering switcher group. 

Twenty-nine percent of persisters in engineering are 
women, whereas 36% of switchers out of engineering are 
women. There were 4% URM students in the persister group 
and 8% in the switcher group. We also found that persisters 
are less likely to be first-generation college-goers (24%) than 
are switchers (32%). Only the gender difference is statistically 
significant (p < 0.05), but these findings suggest that 
persisters are more likely to be male and white and less likely 
to be first-generation college-goers than switchers are. 

Although Figure 2.2-B indicates that engineering overall has 
persistence levels comparable to other majors, the issue of 
persistence remains particularly salient in that women and 
URM students are more likely to switch out of engineering than 
white males are. 


Twenty-nine percent of persisters 
are women, whereas 36% of 
switchers are women. 


Women are more iikeiy to migrate into engineering. 

Another insight from the NSSE data is that, when compared to persisters, a larger proportion 
of migrators into engineering (/.e., students who started in a non-engineering field and were 
majoring in engineering by their senior year) are women (42% of in-migrators vs. 29% of 
persisters). 

Where do the switchers go? Where do engineering in-migrators come from? 

Figure 2.2-E shows where those students who leave engineering go (left) and where those 
coming into engineering come from (right). Over a third of those switching out of an 
engineering major (the non-persisters) take up a science/technology/math major (e.g., 
biology, physics, chemistry, math), and about 14% each go to computer science and busi- 
ness. Combining the science/technology/math and computer science numbers, we see that 
over half of those who switch out of engineering stay in a major that is technical in nature. 
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Figure 2.2-E: Destination majors of those migrating out (switchers) of engineering 
(left) and source majors of those migrating into engineering (right) 
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Looking at those migrating into engineering, over 50% come from the combination of 
science/technoiogy/math and computer science. However, because there are more than 
three times as many students switching out of engineering than are migrating in (182 and 55 
students, respectively, in our NSSE sampie of 11,812), there is a net loss from engineering 
to science/technology/math and computer science. 


2.2.3 Persistence in the Engineering Major: Comparing Coilege Experiences 

Next we consider what we learned about students who matriculated and persisted in an 
engineering major (the 75% indicated by the leftmost bar in Figure 2.2-B), as compared to 
those who switched out of engineering (the other 25%). 


On many measures, engineering persisters and switchers are simiiar. 

On many of the variables investigated through our Longitudinal 
Cohort work, the responses from persisters and switchers were 
indistinguishable during the students’ first years in college. 

Moreover, as students progressed though their undergraduate 
education, the future persisters and switchers ascribed 
comparable importance to and confidence in professional and 
interpersonal skills. (These skills include leadership, 
communication, teamwork, and business ability.) They also 
reported similar levels of financial motivation to study 
engineering and knowledge of the engineering profession, 
comparable frequency of interaction with instructors, and similar 
satisfaction with these instructors, and they displayed similar 


It might be difficult to 
differentiate between a student 
moving away from engineering 
and a student who is likely to 
persist. 
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confidence in solving open-ended problems. (Eri§ etal. 2010) Adding to these Longitudinal 
Cohort findings, the NSSE data indicated other similarities between persisters and switchers. 
They reported comparable grades (McCormick et al. 2010), as well as comparable levels of 
study abroad, community service and volunteer work, participation in research with a faculty 
member, and membership in learning communities (Lichtenstein et al., forthcoming). 
Therefore, by these measures, it might be difficult to differentiate between a student moving 
away from engineering and a student who is likely to persist. 

On some measures, persisters and switchers are different. 

However, persisters and switchers were not similar when viewed through some other 
measures. We saw that switchers were more motivated to study engineering by parents than 
persisters were, whereas persisters seemed more motivated by a high school mentor (Erl§ et 
al. 2010). In addition, switchers appeared to be less confident in their math and science 
skills. This finding should be interpreted in light of other findings from our study indicating 
that it may be interest and confidence in math and science that are drawing students into 
the engineering major to begin with (Kilgore, Chachra, et al. 2009). Switchers also had more 
concerns about financing college than did persisters. Fleming, Engerman, and Williams 
(2006) described the struggle of some Longitudinal Cohort students to remain in an 
engineering major with grade-dependent scholarships. 

Other differences between persisters and switchers emerged during analysis of the NSSE 
data. Persisters were more likely to have a culminating senior experience or co-op and/or 
practicum experience than switchers. In contrast, switchers were more likely to be Involved 
In Independent study work or a self-designed major, and to have completed foreign language 
coursework (Lichtenstein et al. 2010). 


Persisters and switchers differ in intention to compiete an engineering major. 


Even early In their college careers, persisters and switchers were strikingly different in their 
perceptions of whether they will graduate with an engineering degree or not. Figure 2.2-F 
shows the sharp decline (over successive survey administrations) in the intention to 
complete an engineering degree for switchers, compared to 
persisters. Early switchers (students who decide to leave 
engineering sooner than other switchers) were less firm in their 
intentions, even in the first year of college. What is not apparent 
in this figure, however, is that some persisters, despite their 
stronger average intention to complete an engineering major 
relative to switchers, expressed doubt as late as the end of junior 
year of college about continuing as engineering majors (Lichten- 
stein etal. 2007; Matusovich 2008; Matusovich etal. 2008). 


Some persisters expressed 
doubt as late as the end of their 
Junior year about continuing as 
engineering majors. 


Commitment of persisters increases over the four years. 

The significant and upward trend in intention to complete an engineering major among 
persisters shown in Figure 2.2-F was reflected in the structured interviews of these same 
students. In the interviews of students who ultimately persisted In engineering, less than 
half of them, as first-year students, reported being very committed to pursuing an 
engineering major. By the end of their second year, over 80% of them were very committed 
to completing their engineering major (McCain et al. 2007). This is consistent with 
Lichtenstein et a/.’s (2007) observation that students frequently re-evaluate their 
commitment to engineering over the four years of college. 
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Figure 2.2-F: Strength of students' intention to complete an engineering major, with 
1.0 being “Definitely Yes,” 0.5 being “Unsure,” and 0.0 being “Definitely Not.” (Eri§ et 
al. 2010 ) 

Persist: students who had either graduated with an engineering degree or were still majoring in 
engineering (had an engineering major declared on their transcript) as of the summer of their fourth 
year of college (n = 107) 

Switch by Semester 2: students who left engineering major by the end of their second semester {n = 11) 
Switch by Semester 3: students who left engineering major by the end of their third semester (n = 7) 
Switch by Semester 4: students who left engineering major by the end of their fourth semester {n = 14) 



Persist Switch by Semester 2 Switch by Semester 3 Switch by Semester 4 


2.2.4 Factors of Knowledge and Identity that Relate to Persistence 

Two interview-focused analyses explored the relationship between persistence and 
identification with and knowledge of engineering. 

Entering students interested in engineering often have iimited knowiedge of engineering. 

Many students interested in engineering matriculate college with limited knowiedge of and 
exposure to engineering activities. In the sub-group of 32 students in this analysis, we 
frequently heard students describe their enjoyment of and success in high schooi math and 
science. However, we aiso frequently heard that students had little exposure to engineering. 
One sophomore told us that she had not really considered majoring in engineering until her 
senior year in high school. As few as 20% of these first-year engineering students had 
significant exposure to engineering activities (e.g., coursework in engineering during high 
school, engineering internships) prior to matriculating college. Low exposure to engineering 
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prior to college was much more common. Students might have engaged In engineering-llke 
activities but had not been mentored by engineers. (Lichtenstein et al. 2007) 

We also note that many switchers make the decision to leave an engineering major within 
the first two years, the period during which they are taking engineering prerequisites and 
before taking any (or many) engineering courses (Lichtenstein et al. 2007). One potential 
factor In this situation is that students are given little exposure to the many possibilities that 
an engineering career can offer in the first two years, while they are taking math and science 
courses taught outside of engineering departments (Garrison, forthcoming; Jocuns et al. 
2008; Stevens etal. 2008; Stevens eta/.. Engineering student identities, 2005). 

Range of intentions to complete an engineering major 

In our Longitudinal Cohort semi-structured interviews, students revealed a broad range of 
exposure to engineering before and during college. Some students knew they wanted to be 
engineers before graduating high school, while others made an almost off-hand decision to 
enroll In an engineering major. As an example of one who had decided on engineering and 
his future college long before matriculation, Joe said, 

/ guess [I decided to become an engineer] about halfway through high school. I 
really decided that engineering was the thing for me. I took a couple of courses that 
were sort of Introduction to Engineering.. .and I decided that it was the field, and 
then, I sort of started lookin’ around at different places. I was talking to one of my 
teachers who actually works, or interns, at a regional national laboratory, and he 
said they had a lot of good people come out of [my school]. 

Others were less sure as first-year students. Jane, a first-year student, described her prior 
lack of knowledge about the field and ambivalence toward the profession: 

Honestly no, I had no idea what engineering was. I was ]ust like, “Okay, math and 
science school; we got it,” and then like somehow, that ]ust kind of became 
synonymous with engineering— with that definition. They’re like, “Oh you can be an 
engineer,” and I’m like, “Okay, I guess so.” And I only really got a feel for what I’d be 
doing after I got up here [at college]...! don’t know what it [engineering] is. 

Another student, Roger, was uncertain of his commitment to the engineering field and had 
even been interested in majoring in business at a prestigious, private, mid-western university 
prior to enrolling in college. Although his father is an electrical engineer, and although Roger 
had moderate exposure through extracurricular activities in high school, he really was not 
certain what engineers do on a daily basis— or if he wanted to find out. As he said, 

/ see my dad, he’s an engineer. He sits in his cubicle, at his computer all day, typing 
up code and doing stuff. I don’t really want to be doing that, but that’s engineering 
for you. I haven’t really thought about, “Well, after school, what am I gonna be 
doing?” I think it’s sit in a cubicle all day, and I might be doing this, might be doing 
that, and I really don’t know.” 

This work and these quotations suggest that few students— even those who have had some 
prior exposure to engineering— know what engineers do, and this affects their commitment to 
the major (Lichtenstein et al. 2007). As a result, programs that expose students to 
engineering experiences and/or projects early might have a greater chance of both enticing 
students to persist and interesting them in specific sub-fields of engineering. 
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Commitment to engineering depends on students’ identification with engineering activities. 


A student’s reasons for choosing to pursue an engineering degree appear to be related to 
their persistence. Specifically, choosing to major because one identifies with engineering 
and the activities that engineers engage in is positively associated with commitment to 
majoring in engineering. The discussion and quotations in the next two paragraphs are 
drawn from a detailed analysis of interviews of 10 students at one 
Longitudinal Cohort Institution that was part of one of the Ph.D. 
theses based on APS data (Matusovich 2008). (The thesis used a 
framework based on Eccles’ expectancy-value model (Eccles 2005, 

Eccles et al. 1983) and Gee’s (2000) conception of identity.) 


In this sample of 10 students, those with a strong connection 
between their identification with engineering and their perception of 
the activities that engineers engage in are more likely to show 
unwavering commitment to engineering. As third-year student Will 
said. 


Commitment to majoring in 
engineering is reiated to 
identification with engineering 
and engineering activities. 


I have always liked building stuff. I think that’s been the foundation of it. Legos were 
always my favorite toys when I was little. And I’ve always wanted some career where 
I could build stuff, and eventually found out that was called being an engineer. 

In contrast, students with a weak connection between their engineering-related Identity and 
the activities in which engineers engage showed continual renegotiation of their commitment 
to engineering. As fourth-year student Tim said. 

Like the theory, what makes things work, taking things apart, and figuring all this 
stuff out, and all the intricacies, sometimes I just don’t care about that. I’m like, “OK 
so these guys are engineers, that’s what their thing is. Why am I an engineer?" And I 
look at like where I wanna go kind of with my career, and you see this shift away 
from engineering. And I ask myself, “Well so why did I get involved in engineering in 
the first place?" 

Students like Tim demonstrate frequent re-evaluation of whether to stay in engineering 
(Matusovich 2008). Our data also suggest that many students’ decisions about majoring in 
engineering are malleable and that this flexibility even continues up to the point of choosing 
a job (or other opportunity) after graduation (Lichtenstein et al. 2007). 
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2.3 Motivation to Study Engineering 

Motivation is an important factor in looking at the educational pathways of undergraduate 
engineering students. As one theory suggests, people act based on their motivation to fulfill 
basic human needs for autonomy, competence, and relatedness (Ryan and Deci 2000). 
Engineering students are no different. In this section, we examine the question. What 
motivates them to study engineering? The major themes of what we found are reported 
below. 

2.3.1 A Wide Range of Motivational Factors, Most Constant Over Time 

Through our APPLE Survey instrument, we explored six motivational factors, as summarized 
in Table 2.3-A. The first two of these factors— psychological and behavioral— are Intrinsic, in 
that they originate within the individual and are related to enjoyment that is inherent in the 
task or activity itself. The other four— social good, financial, mentor influence, and parental 
Influence— are extrinsic and come from outside of the individual. Except for the behavioral 
motivation factor, which was developed by the APPLES research team, these factors and 
their constituent survey items were adapted from prior education research, as cited in Table 
2.3-A. 

Top motivational factors are behavioral, psychological, social good, and financial. 

The mean scores of seniors on the six motivation factors, as measured with the APPLE 
Survey instrument, are shown in Table 2.3-A. The data show that four factors, on average, 
provided moderate to major motivation for students to study engineering, with psychological 
and behavioral factors at the top. In other words, the top motivators are about how people 
feel when acting and thinking like an engineer. Semi-structured interview data suggest that, 
for some students, an important connection for intrinsic motivation may be pre-college 
behaviors/activities such as “tinkering.” Students in the Longitudinal Cohort talked about 
developing an interest in engineering based upon their childhood Interests in tinkering with 
built objects, sometimes removing broken devices or appliances from the garbage and trying 
to fix them (Garrison, forthcoming). 

Next on the motivation list were factors related to social good and financial security (also see 
Fleming, Engerman, and Griffin 2005). The Longitudinal Cohort Interviews provide a rich 
portrait of students’ feelings about working for the social good. Some students talked about 
using engineering to effect positive change for underserved groups, or working on 
environmental and sustainability issues. As one senior said, “My inclination is that I’ll use 
engineering to try to do development work. ..kind of help the world rather than sit behind a 
desk... I have a real need to find a meaning in what I do with my life.” Another graduating 
student said, “I’m really Interested in energy and the environment and sustainable design. I 
have a passion for service...! really like engineering” (Kilgore, Chachra, et al. 2009). These 
examples also suggest that social good motivation can be linked with intrinsic motivation. 


Mean score 
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Table 2.3-A: Motivational factors considered in APPLES 


Motivational 

factor 

Example 

Psychological 

Studying engineering for its own sake, to experience enjoyment that is 
inherent in the activity, e.g., “1 think engineering is fun,” "1 feel good 
when 1 am doing engineering.” This factor was based on the work of 
Guay et al. (2000). 

Behavioral 

Motivation related to practical and hands-on aspects of engineering, e.g., 
"1 like to figure out how things work,” "1 like to build stuff.” This factor 
was developed by the APPLES research team to capture the hands-on, 
action orientation of engineering. 

Social Good 

Belief that engineers improve the welfare of society, e.g., “Engineers 
have contributed greatly to fixing problems in the world,” "Technology 
plays an important role in solving society’s problems.” This factor was 
borrowed from the Pittsburgh Freshman Engineering Attitudes survey 
(Besterfield-Sacre etal. 1995, 1997, 2001). 

Financial 

Belief that engineering is a financially rewarding career, e.g., "Engineers 
are well paid,” "An engineering degree will guarantee me a Job when 1 
graduate.” This factor was borrowed from the Pittsburgh Freshman Engi- 
neering Attitudes survey (Besterfield-Sacre etal. 1995, 1997, 2001). 

Mentor 

Influence 

Influence of university and non-university affiliated mentors, e.g., “A 
faculty member, academic advisor, teaching assistant or other university 
affiliated person has encouraged and/or inspired me to study 
engineering,” "A mentor has introduced me to people and opportunities 
in engineering.” This factor was developed based on the work of 
Seymour and Hewitt (1997). 

Parental 

Influence 

Parental influences to study engineering, e.g., “My parents want me to be 
an engineer.” This factor was borrowed from the Pittsburgh Freshman 
Engineering Attitudes survey (Besterfield-Sacre etal. 1995, 1997, 

2001). 


Figure 2.3-A: Seniors’ motivation to study engineering on a scale of 0-100, where 
100 indicates major motivator, 66 indicates moderate motivator, and 33 indicates 
minor motivator (Sheppard etal. 2010) 
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We also heard students talk about their perceptions of the financial security of an 
engineering job. (Note that these interviews were conducted before the economic downturn 
that began in 2008.) One senior reflected that “[engineering is] pretty good, like it 
guarantees your iife, that you’re gonna have a good future” (Kiigore, Chachra, et al. 2009). 
Another student was very pragmatic at an eariy stage, saying, “I heard that the petroleum 
engineering school here is an excellent school, and they have like 100% Job piacement, and 
then petroieum engineering has the highest paid engineers of ali the engineers, and so I 
figured uh why not take intro to petroieum engineering?” This same student separated his 
actions from parentai influence and tied future financiai security to a future lifestyle when 
indicating why he chose to major in engineering, offering, “My parents are gonna be happy 
with me and proud of me no matter what I do, so I’m not doing to it make them happy. I’m 
not doing it because society looks at engineers as cooi people, ’cause they don’t. So 
basically— lifestyle, future lifestyle, and money” (Stevens etal. 2007). 

A comparison of transfer and non-transfer students in the Single-School Cross-sectional 
Sample suggests that transfer students are less financially motivated to study engineering 
than non-transfer students (Korte and Smith 2007). 

Mentors and parents are less salient motivators. 

Reiationships, such as with mentors and parents, are iess saiient motivators for engineering 
majors. We do note, however, that parental motivation may be tied to financial issues for 
some students (e.g., “I wanna take care of my parents like they— they’ve been reaiiy good to 
me and uh I wanna be really good to them.”). For others, parental motivation was tied to 
famiiy pride: “I’m a first-generation coilege student, and because this [engineering] is a 
really hard major, graduating college is one thing— it would really be a very big 
accomplishment for me, myself and my family” (Kilgore, Chachra, et al. 2009). 

Motivational factors are interrelated. 

The six dimensions of motivation described in Table 2.3-A are 
interrelated. For example, seniors who pursued engineering for 
its intrinsic vaiue (psychoiogicai and behavioral) also tended to be 
motivated by its potential social value. Mentor motivation was 
positively correlated with social good motivation and 
psychoiogicai motivation, which suggests an important (and 
subtle) role that mentors may play in helping to develop other 
motivational dimensions. 

Certain correlations were only observed by gender. Among senior 
men, parental motivation was correlated with mentor motivation. 

Among senior women, parental motivation and psychological 
motivation were negatively correlated. 

Motivation remains essentially constant over the four undergraduate years. 

The strength and order of the motivational factors in Figure 2.3-A are not appreciably 
different between first-year students and seniors (Sheppard et al. 2010; Korte and Smith 
2008; Eri§ et al. 2007, 2010). Furthermore, an examination of a subset of interview data 
found that the value of becoming an engineer for participants (because of identification with 
engineers) remained stable over the four years. This stabiiity was independent of the extent 
to which the student identified with engineers and engineering. (Matusovich 2008). Overaii, 
our findings suggest that students’ motivation to pursue engineering may take shape eariy 
on in their educational experience and that college reinforces their existing motivation. 


Mentors may play an important 
role in helping to develop other 
motivational dimensions. 


36 


Enabling Engineering Student Success 


Other aspects of motivation: status, portabiiity, and “sticking it out’ 

In our interviews, students gave voice to other factors that are important in their decision to 
major in engineering. These factors include having a degree that was viewed as being highiy 
valued and respected by other fields (Jocuns etal. 2008), working on cutting edge projects, 
having a degree that was potentiaiiy transferable to careers outside of the field of 
engineering, feeling that majoring in math or science would lead to a job with a perceived 
lower status (e.g., in teaching or lab work) when compared to an engineering career 
(Garrison, forthcoming), or beiieving that skiils and confidence in math and/or science would 
directly translate into skiils and confidence in engineering (Kilgore, Chachra, etal. 2009). 

For a few students, the motivation to compiete their engineering degree may be tied to 
“stubbornness” or “doggedness.” This was evidenced in interviews where students stated 
that “I’ve gone too far to turn back now,” and, “I’ve put in a lot of work, and there’s no reason 
to back out whatsoever right now” (McCain et al. 2007). 

2.3.2 Differences Between Men and Women, Between Majors, and Across Years 
Motivation varies with gender and major. 

When we iooked at gender differences in the results summarized in Figure 2.3-A, we found 
that behaviorai motivation was greater among men, whereas mentor motivation was greater 
among women (Sheppard et al. 2010; Kilgore, Chachra, etal. 2009). 

Looking at the effect of major, we saw that senior women majoring 
in BioX engineering (our grouping of all of the bioscience-related 
engineering fields), mechanical engineering (ME), electricai 
engineering (EE), and aerospace engineering (AE) exhibited 
comparable (and high) levels of psychological, behavioral, and 
social good motivation, foliowed by financial motivation. Men 
majoring in BioX engineering showed a similar pattern. In contrast, 
for men majoring in ME, EE, and AE, there was a clear hierarchy: 
behavioral, followed by psychological, followed by social good, 
followed by financial motivation. 

The motivation profiles of women in other engineering majors varied 
with major. For women majoring in industrial engineering (IE) and 
chemical engineering (ChemE), psychologicai, sociai good, and financial motivation were of 
comparable (and high) strengths, foliowed by behavioral motivation. This contrasts with 
women in ME, EE, BioX, and AE, who had comparabie (and high) behavioral, psychological, 
and social good motivation, foilowed by financial motivation. The IE women were much more 
financially motivated to study engineering than their BioX counterparts and much less behav- 
iorally and psychologically motivated than their ME and EE counterparts. (Parikh et al. 2009) 

Motivation is correiated with persistence and satisfaction. 

Among first-year students who took the APPLE Survey, we see that intention to complete an 
engineering major is correlated with their levei of psychological motivation. This suggests 
that students who choose to study engineering for its inherent enjoyment are more likeiy to 
persist in the major. However, psychological motivation is only one dimension of the 
persistence puzzle. Other motivational factors may play key roles in combination with 
psychological motivation. In addition, the individual’s ability to overcome barriers and take 
advantage of supports that occur along their academic pathway can be important. 


Behavioral motivation was 
greater among men, whereas 
mentor motivation was greater 
among women. 
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For seniors, four of the six types of motivation shown in Figure 
2. 3-A— behavioral, psychological, social good, and mentors— are 
correlated with frequency of interaction and satisfaction with 
instructors. These same four types of motivation are also 
correlated with overall satisfaction with college. It may be that 
the motivated students seek out more frequent contact with 
instructors, or that more frequent and satisfying interaction with 
facuity strengthens motivation. It may also be that students who 
are motivated about a particular field and are able to become 
more skilled in that field (through their studies) are just generally 
more satisfied with coliege. 

2.3.3 Identity, Motivation, and Sponsorship 

We discussed above how different types of motivation are correlated in a variety of ways with 
the coliege experience and even with commitment to engineering for first-year and senior 
students, and for women and men. 

Our data suggest that one way students’ personal motivations develop into discipline- 
relevant interests may be through the processes of sponsorship, using Brandt’s (1998) 
terminology. In this view, “intrinsic interest in engineering” is mutually constructed by the 
student and sponsors within the discipline. This concept of sponsorship is useful in looking 
at the development of an engineering identity. Sponsorship, which can be described as 
something ascribed and maintained by others (as in, “You are an engineer”), might come in a 
variety of forms— through faculty, mentors, peers, student support organizations, curricular 
and programmatic structures, or procedures and processes. Sponsorship of student 
interests can play a central role in how and under what circumstances students begin to 
develop an identity as an engineer, and enter, stay in, or leave engineering education. 
(Stevens et al. 2008; O’Connor et al. 2007) 

Identity development as an engineer viewed within a framework of sponsorship 

Aspects of sponsorship in developing (or discouraging) an engineering identity are illustrated 
in two case studies from the Longitudinal Cohort. Adam was initially drawn to engineering 
because of his abilities at math and was successful in his early studies because of these 
skills. As he said, “I like the immediate reward of math, of the black and white. It’s like, if it’s 
wrong, ‘OK, I’ll try it again.’ And then I get it right and it’s like, ‘Yes, I got it right!”’ However, 
he began to founder when he encountered more open-ended engineering problems. As he 
said in his third year, 

That’s like, what the real world is. There’s not a right and a wrong. Whereas a lot of 
my life, it seems, basically can be broken down into that. You get the right answers 
or you do well in class and then you can go to the right college or you can get into the 
right major and stuff. But, I mean, that still applies somewhat, that if you do 
everything right, do everything good, then you get a promotion and stuff. But the real 
world isn’t as right and wrong.. .Instead of me getting it right or getting it wrong and 
reworking it, it’s like it might work out and it might not. You’re not going to know 
right away. So it’s scary... I might be nearing the end of black and white, right and 
wrong phase. 

As Adam engaged in his identity development (which has been dramatically transformed in a 
“scary” way by the unexpected ambiguity of engineering work), he is doing so as a sponsored 
participant in the discipline. This sponsorship results in large part from his early academic 


For seniors, behavioral, 
psychological, social good, and 
mentor motivation are correlated 
with frequency of interaction and 
satisfaction with instructors. 
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performance, which was motivated by interests that he now recognizes as being somewhat 
at odds with how he is coming to see engineering. (O’Connor et al. 2007) 

An example of a lack of engineering sponsorship for a student's interests 

Looking at the second case study, Bryn, a female student, found that her interest in 
considering multiple perspectives in her work was undermined by the competitive, 
individualistic nature of the curriculum. Towards the end of her sophomore year, she 
described her experiences in her pre-engineering classes: 

It just seemed like there was just a different frame of mind and the whole “me 
succeeding,” like, “me, me, me,” and really not wanting to help people, and I didn’t 
understand that, because if I know something, I’m gonna help you figure it out, and I 
would hope that if I didn’t know something, it would be the same way. 

Asked where this different frame of mind comes from, Bryn said. 

Oh man. I don’t know. I think that it just might be the atmosphere of this institution, 
that it’s so big and it’s so competitive, and I think a lot of classes I was taking, people 
were going for some really competitive stuff. ..a lot of those were gonna be weeded 
out. It was always that pressure that, at the next level, someone’s gonna be cut. 

However, she found a different atmosphere in a graduate-level, non-engineering research 
methods class: 

It was so helpful. The grad students were, of course, way more advanced than I 
was.. .We had to work with partners, and the grad student I worked with was just 
really helpful. I mean, like it was just a community more, and we were able to talk. 

There was never a feeling of, like, when you’re in the other classes that are 
competitive, there’s always this underlying feeling of tension, and there wasn’t that 
in that class.. .So there was just, it was just a different community, a different feel. I 
felt a lot more supported. I felt like I could really get good feedback from other 
students. 

In this case, the student describes finding the kinds of experience that she valued, but 
outside of the undergraduate engineering curriculum. In terms of sponsorship, the kinds of 
interests Bryn had do have sponsors, but the sponsors she found were not in engineering. 
(O’Connor et al. 2007) 

These two examples illustrate how sponsorship plays out in students’ lives in a variety of 
ways. Some forms of sponsorship are explicit and intentional, and others are not. 
Regardless, what is particularly important is that sponsorship, and how it serves to validate a 
student’s presence in engineering, may be coming into play more often than we realize in 
drawing students into (or pushing them away from) engineering. 
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2.4 The Engineering College Experience 

Subsection 2.2 compared college experiences of engineering undergraduates with non- 
engineering undergraduates. This subsection focuses more closely on the specific 
experiences of engineering students. We first examine differences between first-year and 
senior students and between male and female students. We then look at issues of students’ 
identification with engineering and highlight some initial findings about transfer students and 
the effects of socioeconomic status. Next, we present some initial thoughts on the 
relationship between motivation and certain confidence measures in defining the 
engineering student’s college experience. The subsection closes with a qualitative 
examination of graduating students’ perspectives on significant learning experiences in 
engineering. 

2.4.1 The College Experience across the Years and As Affected by Gender 

Positive differences between seniors and first-years 

Seniors interacted more with instructors than did first-year 
students, and their courses utilized more project-based 
iearning and working in teams. This trend also held in the 
sophomore and Junior years (Eri§ et al. 2010) and included a 
shift from individual work (in Years 1 and 2) to group work 
(Years 3 and 4) (Stevens et al. 2008). Not surprisingiy, more 
seniors had research, co-op, and internship experiences. They 
were also more active in engineering extracurricular activities 
than were first-years. 

Whereas participation in engineering extracurricular activities 
was greater among seniors than among first-year students, 
participation in non-engineering extracurricular activities was 
comparable between the academic levels. 

Negative differences between seniors and first-years 

As a group, seniors were less satisfied than are first-years with their overall college 
experience. Also, seniors were less satisfied than first-years with their instructors, though 
they report interacting with them more. 

Furthermore, seniors were less academically involved in their courses than were first-year 
students, /.e., they reported being absent or late for class or turning in assignments that did 
not reflect their best work. This was true for both engineering and liberal arts courses and 
took place as a gradual change from the first-year to the senior year, as shown in Figure 
2.4-A. Perhaps we should interpret this apparent decline in academic involvement in iight of 
increased extracurricular participation in engineering activities and research, and even 
increased interaction with instructors. Are students, as they progress through their academic 
career, expanding their ways of iearning about engineering at the expense of high levels of 
curricular participation? Are they learning to optimize their time? Have they iearned what is 
needed to “do schooi,” as their self-reported GPAs are not dropping, in generai? 

While seniors may be more efficient and effective in balancing the complex demands of 
school and “non-school,” we also note that senior men reported a greater sense of curricular 
overload and, specifically, more difficulty in balancing their personal and academic lives than 


Seniors interact more with their 
instructors than do first-year 
students and are more active in 
engineering extracurricular 
activities. 
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Figure 2.4-A: Academic involvement in engineering and liberal arts courses (Eri§ et 
al. 2010) 
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do first-year men. This difference was not present among women; however, women’s sense 
of overload and difficulty with balance exceeded those of men at both the first-year and 
senior leveis. 

We note that students can have a desire for more balance (/.e., pursuits aside from 
engineering studies) than their prescribed program of study will allow (Stevens et al., 
Engineering Student Identities, 2005; Loshbaugh etal. 2006). As an example, one student 
at a technical university in the Longitudinal Cohort viewed the inability to pursue her interests 
in art as a cost or sacrifice associated with being an engineering student (Matusovich 2008). 

Women and men, alike...and different 

In the APS studies, many of the coilege experiences of women and men majoring in 
engineering were similar. They reported similar levels of interaction and satisfaction with 
instructors, similar levels of academic involvement, as well as exposure to engineering 
through co-ops, internships, and research. They also reported similar GPAs. However, we 
observed several noteworthy features of women’s experiences in engineering: 

• Both at the first-year and senior levels, women reported more frequent involvement in 
engineering and non-engineering extracurricular activities than did men. In the case 
of non-engineering activities, women attributed more importance to these activities. 
These differences were statistically significant (p < 0.05). It appears that activities 
outside of the classroom may play a more important role in the lives of undergraduate 
engineering women than of men. (Chachra et al. 2009) 

• Both at the first-year and senior levels, women reported a greater sense of curricular 
overload than did men. In addition, they reported greater pressure to balance their 
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social and academic lives. These differences were 
statistically significant (p < 0.05). However, overload did 
not seem to be a direct result of extracurricular activities; 
we note that, for both women and men, there was no 
simple correlation between overload and extracurricular 
activities. 


Compared to men, women report 
more frequent involvement in 
extracurricular activities and a 
greater sense of curricular 
overload. 


Female study participants on several campuses spoke of 
the importance of doing well to favorably represent their 
gender (Garrison et a/.. Cultural models, 2007; Fleming et 
al. 2008). A fear of representing one’s gender poorly led 
some women to avoid asking for help or to limit from 

whom they sought help— seeking help from other women only. Additionally, some 
women reported being worried that they would have to prove themselves to the men 
in their classes; they would have to demonstrate their knowledge before their ideas 
were considered equal to their male classmates (Garrison et a/.. Cultural models, 
2007; Garrison, Stevens, and Jocuns 2008). 


• Even women who appear to be succeeding as engineering students can be prone to 
seif-doubts. In a series of interviews at a technical university participating in the 
Longitudinai Cohort, we saw that women with consistently high grades can stili doubt 
their engineering ability and be uncertain about practicing engineering. Some women 
redefined what it means to be an engineer to match their perceived abiiities. 
(Matusovich et al., Competence in engineering, 2009) 


Identification with engineering: 

Variations in perceptions of personal cost, enjoyment, and future usefulness 

Some of the college experience may depend on how closely a student identifies with 
engineering. In an analysis of semi-structured interviews at one of the Longitudinal Cohort 
campuses, we saw that students who valued becoming an engineer because it was 
consistent with their sense of self did not see being in engineering school (“learning 
engineering”) as having a high personal cost, in terms of their time and effort. In addition, 
their enjoyment of engineering work stayed the same or increased over their four years of 
coliege studies. 

Perhaps surprisingly, these students’ perceptions of the future usefulness of what they were 
learning in school stayed the same (and relatively low) or decreased over the four years. In 
contrast, among students on one APS campus who identified less with engineering, 
perceptions of usefulness of engineering studies tended to be higher, perceived costs of 
earning an engineering degree higher, and enjoyment of engineering studies lower. 
(Matusovich 2008) 


2.4.2 Other Intriguing and Important Demographic Factors 

Two of the APS data sets generated some intriguing findings relating to the experiences of 
transfer students and the effects of socio-economic status. Although the APS research only 
touched the surface of these fundamental and important areas in American higher 
education, we feel that our initial findings are of broad interest. We include a brief summary 
of the key points below. 
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Transfer students’ experiences 

An analysis of survey and focus group data from transfer and non-transfer students who were 
part of the Single-School Cross-sectional Sample showed that the two groups were 
comparable with respect to their college experiences. However, we did find significant 
differences in other measures: student motivations to study engineering: confidence in 
personal, interpersonal, and problem-solving skills; engagement in non-engineering 
coursework; and satisfaction with their college experience. 

Transfer students reported significantly lower levels of motivation to study engineering for 
financial reasons than non-transfer students. In addition, compared to non-transfer 
students, transfer students reported lower levels of academic involvement for liberal arts 
classes and lower levels of overall satisfaction with their collegiate experience. Transfer 
students also reported lower levels of confidence in personal and interpersonal skills (non- 
engineering skills) and higher levels of confidence in open-ended problem solving than non- 
transfer students. 

Qualitatively, transfer students also spoke highly about their experiences taking classes in 
smaller community colleges before transferring to the larger university. They appreciated the 
smaller class sizes, along with more frequent interaction with faculty, in the community 
college setting. However, they also reported that missing the first year or two at the 
university contributed to the difficulties they encountered integrating into the university 
experience. This probably contributed to their higher levels of dissatisfaction with their 
experiences at the university. (Korte and Smith 2007) 

Socioeconomic status 

Using the survey data from the Broader Core Sample (N = 842), we explored how 
socioeconomic status (SES) might be related to the engineering college experience. SES in 
this preliminary exploration was operationalized to include parental education level and 
perceived family income. 

We found that students in the highest and lowest SES quartiles were indistinguishable from 
one another on the measures of frequency of interaction with instructors, academic 
involvement in engineering and non-engineering classes, motivation to study engineering for 
social good and mentor reasons, and strength of intention to complete their engineering 
degree. 

In contrast, students in the lowest quartile expressed a greater sense of curriculum overload, 
were less satisfied with college and instructors, were less involved in non-engineering 
extracurricular activities, and were less confident in technical skill sets. These same 
students were more motivated to study engineering for financial and family reasons, were 
more involved in engineering extracurricular activities, ascribed more importance to 
professional and interpersonal skills in the practice of engineering, and expressed greater 
intention to continue in engineering after graduation. (Donaldson, Lichtenstein, et al. 2008) 
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2.4.3 The College Experience As Influenced by Motivation and Confidence 

Psychological motivation (motivation to study engineering for its own sake and for the 
enjoyment of it; detailed in Subsection 2.3) and confidence in professional and interpersonal 
skills (self-rated ability in leadership, communication, teamwork, business ability, social self- 
confidence, etc.) were significant predictors of students’ post-graduation plans. These two 
variables are discussed in Subsection 2.7 in the context of students’ future plans, but their 
importance prompted us to also explore what (if any) influence they might have on students’ 
college experience. 


Table 2.4-A: Definitions of groups of seniors based on levels of psychological 
motivation (M) and confidence in professional/interpersonal skills (C). For the m/c 
notation, bold capital letters indicate "above mean” and lowercase letters indicate 
"below mean” for the relevant measure. 


Group 

label 

Psychological 
motivation (M) 

Confidence in 
professional/ 
interpersonal skills (C) 

Group 

characterization 

M/C 

At or above mean 

At or above mean 

Fligh involvement 

M/c 

At or above mean 

Below mean 

Average involvement: 
engineering focused 

m/c 

Below mean 

At or above mean 

Average involvement: 
non-engineering focused 

m/c 

Below mean 

Below mean 

Low involvement 


Focusing only on APPLES seniors (n = 1098), we defined four distinct groups using the mean 
for each of the two variables as the cut-off between the groupings. The four groups are 
defined in Table 2.4-A, and roughly a quarter of seniors fell into each of the groups. Groups 
are also labeled with an abbreviated “m/c" notation, where M stands for psychological 
motivation and C stands for confidence in professional and interpersonal skills. Bold capital 
letters indicate “above mean” and lowercase letters indicate “below mean" for the relevant 
measure. 

On some measures, the groups are the same. 

On a few measures, there were no significant differences among the four groups. For 
example, self-reported GPA, sense of curriculum overload, and motivation to study 
engineering for financial or parental reasons did not vary by group. 

Synthesizing the differences to characterize each group 

We next consider differences in college experiences and motivational factors by group. 

Within each group, we look at how variable scores compared with the average across all 
seniors. We also note when scores were at the extremes of a variable’s range, referring to 
“top" or “bottom scores.” 
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High psychological motivation, high professional/interpersonal confidence (M/C) 

The seniors in this group were consistentiy at the top in extracurricular participation and self- 
reported knowledge of engineering. Extracurricular participation includes both engineering 
and non-engineering activities, as weil as co-op, internship, and research experience. That 
students in this group were at the top in these measures suggests that these students 
developed a more complete picture of the nature of engineering work. These students were 
aiso among the most motivated, with top scores on the behavioral, social good, and mentor 
motivation factors. Also noteworthy is that these students were top in interacting with 
facuity. Given this high level of participation in multiple ways, we refer to these students as 
the High Involvement Group. 

Low psychological motivation, low professional/interpersonal confidence (m/c) 

The seniors in this group were consistentiy at the bottom in self-reported extracurricular 
participation and knowledge of engineering. These students were among the least 
motivated, with scores on behavioral, social good, and mentor motivation variables at the 
bottom. Further, these students reported interacting less with faculty than did students in 
the other groups. Given this iow level of participation in muitiple ways, we refer to these 
students as the Low Involvement Group. 

High psychological motivation, iow professionai/interpersonai confidence (M/c) 

The seniors in this group were near the mean in seif-reported extracurricular participation 
and knowledge of engineering. They reported interacting with facuity at the average ievel. 
These students were among the most motivated, with high scores on behavioral, social good, 
and mentor motivation variabies, on par with the High Involvement Group. Given their 
average ievel of participation and high ievel of motivation to study engineering, we refer to 
these students as the Average Involvement: Engineering-focused Group. 

Low psychoiogical motivation, high professionai/interpersonai confidence (m/C) 

The seniors in this group were near the mean in extracurricular participation, except non- 
engineering extracurricular activities, where they were at the top. They also reported average 
knowledge of engineering and interaction with faculty. These students were among the least 
motivated to study engineering, and as such were similar to students in the Low Invoivement 
Group. Given their average ievel of overall participation, high level of participation on non- 
engineering activities, and low level of motivation to study engineering, we refer to these 
students as the Average Involvement: Non-Engineering-focused Group. 

Demographics by group 

The proportion of women varied little by group with one exception: women comprised a 
greater proportion of students in the group with below-mean psychologicai motivation and 
interpersonai and professional confidence group (the m/c group). In addition, there were 
proportionally more underrepresented racial/ethnic minority men and women in the two 
groups with high psychologicai motivation, as compared to the two groups with below-mean 
psychological motivation. 

On average, students in the m/C group reported higher family income (suggesting higher SES 
in this group), as compared to students in the two groups with lower confidence in 
interpersonai/professional skills (M/c, m/c). Students in the M/C group feli in the middie of 
the famiiy income spectrum. 
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An emerging picture of invoivement 

The picture that starts to emerge is that students in the High Invoivement and Low 
Involvement Groups are “opposites" on many measures. Students in the High Involvement 
Group were highly engaged with engineering and non-engineering activities, both inside and 
outside of the classroom, more motivated to study engineering, and more confident in their 
math/science skills (in addition to professional/interpersonal skiiis). In contrast, students in 
the Low Involvement Group were far less involved with engineering and non-engineering 
activities and were less motivated, less confident, and iess satisfied with the overall college 
experience. We note however that students in these two groups were the same on key 
course-related measures: self-reported GPA, academic involvement in engineering classes, 
and sense of curricular overload. Thus, students in the Low Invoivement Group, relative to 
those in the High Involvement Group, may be hard for faculty to spot, as they do not 
differentiate themseives by these course measures (whereas they do by out-of-ciass 
measures). 

Students in the Average Involvement Groups were between students in the High and Low 
Involvement Groups on many measures. They reported average involvement in their 
engineering courses and interaction with instructors. However they were different from one 
another in severai important ways. First, the students in the Engineering-focused Group 
focused more of their extracurricuiar invoivement on engineering activities, whereas those in 
Non-Engineering-focused Group focused more on non-engineering activities. Secondly, the 
students in the Engineering-focused Group were highly motivated to study engineering for 
social good reasons (in addition to psychological motivation), whereas those in the Non- 
Engineering-focused Group indicated far less social good motivation. 

2.4.4 Qualities of Significant Learning Experiences: Student Perspectives 

When asked during interviews to talk about their most significant learning experiences whiie 
in coiiege, graduating seniors at the iarge, public research university identified severai 
qualities that made a difference in their learning (Kilgore, Jocuns, and Atman, Significant 
learning experiences, forthcoming): 

• Encouraged or even required self-directed learning 

• Hands-on and/or clearly applicable to the “real world” 

• Enabled students to integrate a lot of diverse knowledge, to see the “big picture” 

• Provided intrinsic motivation by empowering students to “own” the experience 

• Challenged students 

For example, Kara, a mechanical engineering student, described how her senior design 
project incorporated ail of the above qualities. First, the project demanded a great deal of 
self-direction from Kara and her teammates, who were “left to our own devices a iot.” 
Directing their own iearning and progress on the project entaiied “incorporating everything 
that we had learned up until that point,” and aiso learning some facets of engineering project 
management. Kara felt that the complexity of the “logistics and the running around and the 
getting peopie together and. ..keeping people on task" added realism to the project, and thus 
prepared her for future empioyment. Furthermore, Kara reflected how the project 
“incorporated everything that we had iearned up until that point” and thus was both a 
challenge and an opportunity. The team “wouid have iiked a iittle more direction,” but Kara 
aiso acknowledged that the self-directed iearning required by the senior project aiso 


46 


Enabling Engineering Student Success 


empowered them to own the project, as “in the end, it was good for us to have that 
experience of trying to make it on our own and make the decisions by ourselves.” 

In addition to describing the qualities discussed above, several students explained that these 
learning experiences were significant to them because they entailed a transformation in how 
students thought about themselves, engineering, and/or the world. Kara’s senior design 
project caused her to broaden her conception of engineering work and thus transformed her 
vision of herself in her future profession. 

Kara’s reflections on significant learning were not unique. All 15 seniors whom we 
interviewed at the large, public research university identified some or all of above qualities in 
describing what made a learning experience significant for them. Three of these students 
cited an excellent teacher or mentor who cultivated their interests and abilities by 
incorporating some of the above qualities into learning experiences. These students’ stories 
suggest that learning experiences can be designed to encourage self-directedness, 
applicability, knowledge integration, student ownership of the experience, and challenge, and 
that teachers can play a role in creating environments in which significant learning takes 
place. 
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2.5 Engineering Knowledge, Conceptions, and Confidence 

This section discusses a wide range of findings concerning undergraduate engineering 
students’ understanding of, conceptions of, and confidence in engineering in generai. (Note 
that Subsection 2.6 separateiy focuses on design as a specific aspect of engineering.) We 
begin with a discussion of discipiinary knowiedge and a description of students’ increasing 
use of engineering ianguage, foiiowed by findings about how and where APS students report 
iearning about engineering. Then, we describe student confidence ieveis for severai key 
engineering skiiis and how they change over students’ coiiege years. We finish with some 
chaiienges: resuits from one of our studies reveaiing seniors’ difficuities in understanding 
certain fundamentai engineering concepts. 

2.5.1 What Counts as Engineering: Accountabie Discipiinary Knowiedge 

Students’ understanding of engineering discipiinary knowiedge changes over time. 

in our anaiyses of semi-structured interviews and fieid observations with Longitudinai Cohort 
students, we described discipiinary knowiedge as an important dimension of engineering 
undergraduate deveiopment (Stevens et al. 2008). Our study of discipiinary knowiedge 
empioyed an ethnographic approach and examined how context and interpretation affect an 
individuai’s notions of what counts as discipiinary knowiedge. This approach recognizes that 
different peopie conceive of engineering in different ways, and that these conceptions 
change over time. We use the term accountable disciplinary knowledge to refer to 
phenomena that are interpreted as counting as engineering knowiedge. For instance, 
students enroiied in an introductory engineering course might interpret the presence of 
design probiems and teamwork as indications that both design and coiiaboration are 
important skiiis in engineering. 

Some students struggle with the shift from ‘book probiems” to open-ended probiems. 

For some undergraduate engineering majors, accountabie discipiinary knowiedge and 
images of engineering shift substantiaiiy during their four years of study. Emphasis on 
soivingformuiaic, weii-defined probiems with singie correct answers in iecture-based, first- 
year courses is repiaced by a focus on open-ended probiems without ciear, “biack-and-white” 
answers in upper-ievei project courses. Some students struggied with this shift in probiem- 
soiving styie. in addition, eariy courses rareiy conveyed the range of possibiiities offered by 
an engineering career (Stevens et al. 2008; Garrison, forthcoming). Students reported 
differences between their engineering experiences in courses vs. during internships and 
co-ops (Stevens et al. 2008). 

Use of engineering-specific ianguage increases during the undergraduate years. 

Muitipie data sets coiiected from the APS Longitudinai Cohort refiect a shift in engineering 
students’ use of ianguage as they progress in their undergraduate studies. The Longitudinai 
Cohort survey findings discussed in more detaii in Subsection 2.6 describe a shift in 
vocabuiary, specificaiiy pertaining to design. This anaiysis showed that senior and first-year 
students associated different activities with engineering design, with seniors tending more 
toward engineering-specific activities, such as identifying constraints and iterating, in 
contrast, first-year students were more iikeiy to associate design with more generai activities, 
such as communicating, pianning, and visuaiizing (Atman, Kiigore, and McKenna 2008). 
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Jocuns and Stevens’ detailed case study of one Longitudinal Cohort student’s four-year 
experience describes how he grew to use more technical, engineering-specific vocabulary. 
Beyond vocabulary, he also became more fluent with different processes and techniques for 
communicating in engineering, e.g., use of diagrams, theories, and design. (Jocuns et al. 
2008) 

The remainder of this section focuses more directly on findings concerning students’ 
conceptions of engineering and the kinds of experiences that affect these conceptions. 

Many of these findings are based on survey data and detail the skills and knowledge that 
students consider important in engineering practice. 

2.5.2 How Students Learn about Engineering 

Students’ knowledge of engineering does grow from first to senior year. 

As expected, seniors reported having learned more about engineering work since starting 
college than do first-year students. Students’ knowledge of engineering is low at first and 
then increases over time, but their knowledge of engineering can be limited by a lack of 
exposure to the profession. As mentioned above, students encounter different images of 
engineering knowledge over time and must adjust to these changing images (Stevens et al. 
2007, 2008; Korte & Smith 2008; Jocuns et al. 2008). 

When asked about their knowledge of engineering prior to college, seniors reported lower 
levels than did first-year students. This might partly reflect the knowledge of engineering that 
seniors have gained since starting college; compared with first-years, seniors might more 
easily recognize (in retrospect) what they did not know about engineering before college. 

Co-ops and internships build knowledge of engineering. 

Students attributed gains in knowledge of engineering practice to a number of sources, and 
seniors cited more sources than did first-years. Seniors, more so than first-years, reported 
this knowledge coming from co-op and internship experiences (“work-related experiences” in 
Figure 2.5-A). For seniors, co-op and internship experience was the most commonly reported 
source, followed by school-related experiences. School-related experiences (e.g., professor, 
class) were cited by approximately 60% of first-year and senior students. Looking at one 
particular school-based practice, we found that, for first-year students (but not for seniors), 
team-based project work was correlated with gains in engineering knowledge. 

Many seniors did not perceive gaining knowledge of engineering from school-related 
experiences. 

That 40% of seniors did not identify their school-related 
experiences as adding to their knowledge of engineering 
practice Is perhaps surprising, particularly given the increase in 
team-based project learning happening at the senior level (as 
described in Section 2.4). It may be that co-op and Internship 
assignments make engineering in the academic setting seem 
Just that: academic, relative to their co-op and internship work 
(Stevens eta/.. Engineering Student Identities, 2005). This 
interpretation is consistent with what Matusovich (2008) found 
among students at one of the Longitudinal Cohort campuses. 

These seniors valued becoming an engineer because it was 


40% of seniors do not identify 
their schooi-reiated experiences 
as adding to their knowiedge of 
engineering practice. 
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Figure 2.5-A: Percentages of students who report gaining knowledge about 
engineering from various sources, with asterisks indicating statistically significant 
differences between first-years and seniors (*p < 0.05, ***p < 0.001; 869 first- 
years, 1130 seniors) 



Work-related Personal contacts*** School-related Other source* 

experiences*** experiences 

Source of engineering knowledge 


□ First-years □ Seniors 


consistent with their sense of self; their perceptions of the future usefulness of what they 
were learning stayed the same (and relatively low) or decreased over the four undergraduate 
years. 

Are capstone projects not realistic and too late in the curriculum? 

Furthermore, we note that most of the students we studied completed capstone or senior 
design project courses in their last year of school. Many students felt unprepared for the 
kind of work involved in these capstone projects, which were designed to be culminating 
senior-year experiences. Many felt they had not done enough design work prior to the 
capstone. Some students felt that their projects were not representative of work they would 
ever do in industry and consequently viewed their capstone experiences as inauthentic. In 
addition, they wished the capstone had occurred earlier in their respective programs or had 
been introduced into the curriculum in a more progressive manner. (Garrison, forthcoming; 
Adams et a/., forthcoming) We saw that more intentional bridging might be required to 
connect students’ work experiences with in-class experiences (a point we will return to in 
Subsection 2.10). 

Students recognize different skills as important in engineering. 

Both first-years and seniors perceive math and science skills as being more important in 
engineering practice than professional and interpersonal skills (e.g., leadership, 
communication, teamwork). This finding is not surprising, given the central role that interest 
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and skill in math and science play in recruiting students to engineering in the first place, not 
to mention their prominent and early placement in a typical engineering curriculum. 

Comparing first-years and seniors, we see that seniors rated math and science skills as being 
less important than did first-year students. There may be several reasons for this difference. 
Based on experience with larger projects (e.g., in capstones or internships/co-ops), seniors 
may be seeing math and science as Just one of many skill sets needed in engineering 
practice. First-year students, whose perceptions might largely be based on high school 
advising, may believe that math and science are more central in engineering. In addition, the 
difference might reflect the tendency for engineering students to take more math and 
science courses in the first year, relative to senior year. The decrease in importance might 
also reflect differences in how math and science are practiced in school and how they are 
practiced in the workplace. In a workplace context, seniors might not recognize math and 
science as they learned it in school, resulting in a lower rating of importance, at least among 
seniors with industry exposure. 

We also see seniors reporting the importance of professional and interpersonal skills at a 
level comparable with (if not slightly less than) what first-year students reported. It was 
surprising to us that seniors did not rate professional and interpersonal skills more highly, 
given their significantly greater participation in co-op, internship, project-based learning 
experiences, and engineering extracurricular activities, i.e., all activities with a considerable 
amount of social interaction. 

Learning about engineering is mostiy similar among women and men. 

Women and men report similar gains in learning about engineering from their first to senior 
year, and cite similar sources for these gains. However, we observed one noteworthy gender 
difference: Women report professional and interpersonal skills as being more important 
than do men (and this gender difference is greater among seniors than among first-years). 

Importance of and preparedness with engineering skills and knowledge 

When surveyed during their senior year, engineering students in the Longitudinal Cohort were 
asked two additional questions concerning engineering skills and knowledge. The first 
question provided the respondents with a list of engineering skills and knowledge items and 
asked them to select the five most important (Figure 2.5-B, with an alternative visualization 
in Figure 2.5-C). The list drew items from the ABET Criterion 3 program outcomes list (ABET 
2009) and the National Academy of Engineering report. The Engineer of 2020 (NAE 2004). 
The second question asked students to rate their level of preparedness (in the context of 
engineering practice) with each of the ABET/2020 items. 

Seniors were most likely to identify Problem solving, Communication, Teamwork, and 
Engineering analysis as important items, with at least 50% of them including each of these in 
their list of five most important items. The least selected items included Contemporary 
issues. Societal context. Global context, and Conducting experiments. Less than 5% of 
seniors selected each of these items. Students’ self-rated preparedness responses mostly 
mirrored their importance responses. Most seniors reported higher preparedness with items 
Teamwork, Problem solving, and Communication. On average, seniors reported the lowest 
levels of preparedness with items Contemporary issues. Business knowledge. Global context, 
and Societal context. (Yasuhara, Perceived importance and preparedness, 2008) 
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Figure 2.5-B: Engineering skills and knowledge items and the percentage of 
Longitudinal Cohort seniors who selected each among their set of five most 
important items (n = 109) 
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Figure 2.5-C: Word cloud (created at http://www.wordle.net/) of engineering skills 
and knowledge items, with larger items being more frequently selected by 
Longitudinal Cohort seniors as one of the five most important items (n = 109). 
Differences in darkness are arbitrary and serve oniy to ciarify separation among 
adjacent items. 
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On the one hand, these ABET/2020 findings suggest that graduating engineering students 
appreciate and feel prepared in certain professional skills like communication and 
teamwork. In contrast, low prioritization and feelings of preparedness with contextual 
knowledge seem at odds with the global and complex nature of modern engineering work. 
This interpretation is moderated by the possibility that students considered the importance 
of contextual knowledge as part of another Item, Ethics, which 40% of seniors included in 
their lists of five most important items. 

There were important differences in the phrasing and format of the APPLES and Longitudinal 
Cohort ABET/2020 survey questions concerning the importance of certain skills. These 
differences precluded direct comparison of responses, but the findings are qualitatively 
similar in that students in both samples placed high importance on both math/science and 
professional skills. As for the APPLES finding that seniors rate math/science skills as more 
Important than professional/interpersonal skills, the ABET/2020 findings might simply 
reflect that certain professional/interpersonal skills (communication and teamwork) are 
considered important, but others (e.g., leadership and business knowledge) are considered 
less important. (The Longitudinal Cohort ABET/2020 questions asked about each of these 
specific professional/interpersonal skills.) 

2.5.3 What Does Confidence Look Like? What Contributes to it? 

Through the APPLE Survey instrument, we explored three aspects of students’ confidence: 
confidence in their math and science skills. In their professional and Interpersonal skills, and 
In open-ended problem solving. This subsection describes differences between seniors and 
first-year students in these aspects of confidence, as well as the factors that seem to 
contribute to their development. 

Not all confidence levels are equal; women’s confidence lags in some areas. 

The APPLES engineering majors generally have an above-average level of confidence 
(relative to their non-engineering peers) in their open-ended problem solving skills, math and 
science skills, and professional and Interpersonal skills, though confidence in open-ended 
problem solving is greatest (Figure 2.5-D). Among seniors, confidence levels in professional 
and interpersonal skills were comparable among women and men. At the same time, 
women were less confident than men In their open-ended problem solving and math and 
science skills. Noteworthy Is that the confidence gap in open-ended problem solving among 
senior women and men was greater than the gap among first-years, suggesting that college 
experiences are affecting the development of confidence differently for women and men. 

Students exhibit low confidence in professional and interpersonal skills. 

Although students’ confidence in these three measures was 
generally high, students rated themselves least confident In 
professional and interpersonal skills. This was true regardless 
of gender, cohort or underrepresented raclal/ethnic minority 
(URM) status. Seniors’ lower confidence in the critical area of 
professional and interpersonal skills (when compared to open- 
ended problem solving and math and science skills) may 
strengthen the argument that engineering is not doing a 
particularly good Job In this domain. As we will see In 
Subsection 2.7, which describes students’ plans for the future, 
we may be losing some top students to non-engIneerIng paths 
as a result of this relative lack of confidence. 


Students rate themselves least 
confident in professional and 
interpersonal skills. 
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Figure 2.5-D: Confidence levels of first-year (top) and senior (bottom) engineering 
majors in the Broader National Sample (APPLES), by gender, with asterisks indicating 
statistically significant gender differences (*p < 0.05, **p < 0.01, ***p < 0.001; 
311 and 557 first-year women and men, 326 and 795 senior women and men, 
respectively) 



Open-ended problem-solving* * * Math and science* * Professional and interpersonal 

skills 


Confidence measure 


■ Women DMen 


Confidence in math and science skiiis remains constant. 

Confidence in math/science skills (relative to peers) remained constant at an above-average 
level for men and women when first-year and seniors were compared. One possible 
explanation for this lack of growth in confidence is that seniors realize there is more math to 
learn. Another is that they are seeing that, relative to their peers, their math skills remain 
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comparable. Alternatively, students might realize that the math they have learned in school 
may not be the math needed in practice. 

Confidence in math and science skills was predicted by self-reported GPA. We cautiously 
interpret this relationship as indicating that this confidence is grounded in school-measured 
academic performance. The finding that being female is a predictor of lower confidence in 
math and science skills is consistent with another segment of APS research at one of the 
Longitudinal Cohort institutions, where women with high grades could still doubt their 
engineering ability and revise their positive beliefs about their competence in engineering 
(Matusovich eta/.. Competence in Engineering, 2009). 

That gender and self-reported family income (one indicator of 
socioeconomic status) were predictors of confidence in math and 
science skilis is of concern. At the same time, that race/ethnicity 
does not seem to have influenced confidence in these key skills 
is noteworthy. Frequency of interaction with facuity, involvement 
in research, engineering extracurricular activities, and exposure 
to engineering through co-ops, internships, and work experience 
ail had no predictive power with respect to confidence in math 
and science skills. Future work might address the question of 
why confidence in a set of skilis that are so centrai to engineering 
was not positively predicted by any of the components of an 
engineering education that one might expect would contribute to 
the development of a student’s confidence. 

Non-engineering factors iargeiy contribute to confidence in interpersonai skiiis. 

As many educators would hope, confidence in professional and interpersonal skills was 
greater among seniors than first-year students. At the same time, students’ perceived 
importance of these skiiis was not greater among seniors than among first-year students. 

We are ieft to wonder why these skills were not deemed to be more important, given seniors’ 
greater co-op, internship and research experiences and increased project-based learning 
courses. 

Confidence in professional and interpersonal skills among seniors was predicted by family 
income, one indicator of socioeconomic status (SES). Therefore, all things being equai, a 
senior from a higher family income will be more confident than a senior from a lower income. 
Family income was also a predictor of confidence in professional and interpersonal skills in 
the first-year model, albeit a weaker predictor than in the senior-year model. 

We also found that confidence in professionai and interpersonal skills was predicted by 
invoivement in non-engineering extracurricuiar activities. The regression model for first-year 
students suggests that frequency of interaction with faculty is important in addition to 
participation in non-engineering extracurricular activities. 

Confidence in professional and interpersonal skills was weakly predicted by invoivement in 
engineering research or engineering extracurricuiar activities, and was not predicted by 
exposure to engineering through co-op, internship or work experience. Our findings suggest 
that these engineering-focused activities are not engendering the same types or level of 
professional and interpersonal confidence that involvement in non-engineering activities is. 
This weak effect might be because more sociaily confident students are drawn to non- 
engineering activities to a greater extent than are less socially confident students. 
Additionaily, engineering-focused extracurricular activities might not contribute to 
development of social skills concurrently with technicai expertise, suggesting room for 
improvement of these activities. 


Gender and self-reported family 
income, but not race/ethnicity, 
predicted math/science 
confidence. 
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Possible explanations for differences in perceived importance and confidence in key skills 

At the very least, the differences described above with respect to confidence in and 
perceived importance of key engineering-related skills (by gender, race, and class standing) 
reinforce that “the college experience” is complex and heterogeneous, affecting a variety of 
perception and confidence outcomes. We expect that the following (or some combination) 
may be happening as part of these coliege experiences: (1) various groups experience 
different curricular and extracurricular activities with which to build their understanding and 
confidence; (2) various groups experience the same curricular and extracurricular activities 
but internalize them in different ways; and/or (3) various groups come to college with 
different pre-college experiences on which to overlay their college experience. Teasing apart 
and testing these “candidate hypotheses" on differences may be useful in gaining a more 
fundamental understanding of how college affects students in different ways and how 
education can be improved for all students. 


2.5.4 Misunderstanding Key Concepts 

Mastering key technical concepts in ways that can be brought to bear in engineering analysis 
is criticai to becoming an engineer. The Difficult Concepts Study employed specially 
developed interview protocols to investigate student learning of various key concepts in 
engineering mechanics and electrical circuits. 


Even graduating seniors misunderstand some key engineering concepts. 


During hour-long interviews at one of the Longitudinal Cohort institutions, two groups of 
graduating seniors (with majors in electrical and civil/mechanical engineering, respectively) 
did not understand concepts such as force and voltage well 
enough to explain them. Students typically thought of 
phenomena like force and voltage as substances instead of as 
processes or interactions. For example, students responding 
to a question about a free body diagram might say that tension 
is “a force inside a rope,” seeing force as a property of the 
rope, rather than the interaction between two or more bodies. 

Electrical engineering students wouid talk about voltage as 
being the property of a particular location, not the difference in 
electrical charge between two points. (Streveler et a/.. 

Identifying and investigating difficult concepts, 2006) 


Seniors [in ME and EE] did not 
understand concepts such as 
force and voltage well enough to 
explain them. 


Other ongoing work in thermal science, particularly heat transfer, suggests that chemical and 
mechanical engineering students are also misapplying the process vs. substance schema by 
confusing rate with amount (Streveler et al. 2008). Taken together with the APS 
investigations, these studies suggest that some students from chemical, mechanical, and 
electrical engineering use substance-based models for processes. Thus, helping students to 
create more accurate mental models that represent processes correctly (not as substances) 
may help greatly in many areas of engineering education. (Streveler et al., Identifying and 
investigating difficult concepts, 2006) 
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Faculty are often unaware of misunderstandings and the difficulty of these concepts. 


During interviews with faculty to develop the Difficult Concepts 
Interview protocols, we observed that engineering faculty often 
thought that graduating students did understand these 
concepts. Furthermore, faculty typically did not rate these 
concepts as difficult for students to understand at all. 

(Streveler et a!., Identifying and investigating difficult concepts, 
2006) Further research could illuminate what makes these 
Ideas conceptually so hard to master. Thinking further about 
engineering practice, we need to examine the extent to which 
students are able to pass courses In engineering fundamentals 
without a “working” knowledge of fundamental concepts. 


Are students able to pass 
courses in engineering 
fundamentals without a 
“working” knowledge of 
fundamental concepts? 
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2.6 Engineering Design Knowledge, Conceptions, and Confidence 

A variety of qualitative and quantitative data were collected from the Longitudinal Cohort to 
examine engineering design from muitiple perspectives. The findings discussed in this 
section cover conceptions of design, confidence in design abiiity, experience with design 
activity, and approaches to open-ended design problems. Longitudinal analyses provide an 
understanding of how engineering students change with respect to these topics during the 
course of four years of undergraduate education. As discussed later in this subsection, 
comparative analyses of the Longitudinal Cohort data sets listed in Table 2.6-A show gender 
differences in a variety of design-related measures. 


Table 2.6-A: Data sets whose analyses are discussed in this subsection, with year of 
collection (Longitudinal Cohort) 


Data set 

First 

year 

Sophomore 

year 

Junior 

year 

Senior 

year 

Most important design activities survey 
question 




•/ 

Confidence, experience, and 
preparation with design survey 
questions 




•/ 

Midwest floods design task 

vC 


•/ 


Street crossing design task 


✓ 


•/ 

Playground design survey question 




•/ 

Microchip factory design task 




•/ 


2.6.1 Conceptions of Engineering Design 

Conceptions of engineering design shift during the undergraduate years. 

In first- and senior-year surveys, students were asked to identify the six design activities they 
considered most important, selecting from a comprehensive list of 23 items (Figure 2.6-A 
and Figure 2.6-B). Overall, most students selected Understanding the problem in both their 
first and senior years of study. In the first year, other more frequentiy selected items 
included Communicating, Planning, and Brainstorming, all of which were selected by at least 
40% of participants. By their senior year, student conceptions of engineering design appear 
to have shifted. More seniors selected Identifying constraints and Iterating as important 
design activities than as first-year students. Conversely, fewer students selected Planning, 
Visualizing, and Communicating in their senior year, compared to their first year. These 
findings might reflect a shift toward thinking about design in terms of more engineering- 
specific ianguage. (Atman, Kilgore, and McKenna 2008) 
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Figure 2.6-A: Percentage of seniors selecting design activities as one of the six most 
important, all APS respondents in longitudinal subsample (n = 89) 
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Figure 2.6-B: Change in percentage between first and senior year of participants 
selecting design activities as one of the six most important, all APS respondents in 
longitudinal subsample (n = 89; *p < 0.05, **p < 0.01) 
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Conceptions of engineering design vary with gender and institution. 

Comparisons of women’s and men’s responses to the same survey questions reveal some 
gender differences in conceptions of engineering design. As first-years, women were more 
iikely to select Seeking information and less likely to select Buiiding and Prototyping. Gender 
differences in the students’ senior-year responses were similar, with women more likely to 
select Goai setting and less likely to select Buiiding. (Chachra et a/. 2008) 

While students at the Longitudinal Cohort institutions were largely similar in their 
conceptions of engineering design, seniors’ responses exhibited some institutional variation 
for items Communicating and Prototyping. Among the four Longitudinal Cohort institutions, 
students at the public research university devoted to engineering and applied science were 
the most likely to select Communicating (75%, compared with as low as 30% at the private 
research university). Students at the private research university, however, were the most 
likely to select Prototyping (48%, compared with no more than 13% at each of the other three 
institutions). These differences might reflect the curricular and cultural emphases particular 
to the respective institutions. (Atman, Kilgore, and McKenna 2008) 
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2.6.2 Confidence, Experience, and Preparation with Design 

Another set of survey questions, administered in the sophomore and senior years, examined 
student confidence, experience, and preparation with respect to a set of eight specific design 
activities: 

• Defining what the problem really is 

• Searching for and collecting information needed to solve the problem 

• Thinking up potential solutions to the problem 

• Detailing how to build the solution to the problem 

• Assessing and passing judgment on a possible or planned solution to the problem 

• Comparing and contrasting two solutions to the problem on a particular dimension 
such as cost 

• Selecting one idea or solution to the problem from among those considered 

• Communicating elements of the design in sketches, diagrams, lists, and written or oral 


In both the sophomore and senior years, students on average 
described their confidence to do each of the design activities as 
Good or Very good. Students on average also reported that they 
engaged in each design activity as part of their coursework 

approximately 2-3 times per week to 1-2 times per month. Finaliy, students rated as Good 
to Very good the preparation from their classes to do each of the design activities. (Morozov 
et a/. 2008) 

Men report higher confidence and course preparation with design than women. 

Both as sophomores and seniors, men generally indicated higher leveis of confidence and 
course preparation than women for engaging in engineering design activities. Gender gaps 
were more pronounced in sophomores’ responses. Compared to women, men reported 
significantiy higher confidence with probiem definition, generating ideas, and modeling as 
sophomores and modeling again as seniors. Findings for course preparation were similar, 
and men also reported significantly better course preparation with modeling, evaluation, and 
decision-making in the sophomore year and decision-making again in the senior year. These 
gender gaps are especially notable in iight of the absence of gender differences in the 
frequency of engagement with the design activities in coursework (in either year). Taken 
together, these findings suggest that there was a gender difference in the quality of design 
education (if not quantity) that students in this sample received. (Morozov et al. 2008) 

2.6.3 Consideration of Context During Design 

We explored the extent to which APS students consider contextual issues while engaged in 
engineering design by presenting them with a variety of open-ended design problems over 
the course of their undergraduate education. Examples of contextual issues include 


reports 


For each of the above activities, we asked Longitudinal Cohort 
students to (a) rate their confidence in their ability to engage in 
the activity, (b) indicate the frequency of their engagement with 
the activity in their courses, and (c) rate how well they believe 
their courses are preparing them to engage in the activity. 


Students reported engaging in 
design activities as part of their 
coursework approximately 2-3 
times per week to 1-2 times per 
month. 
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APS explored the extent to which 
students consider contextual 
issues during design (e.g., 
environmental, economic, and 
social factors). 


environmental, economic, and social factors relevant to a design 
problem. Two of the problems were presented in alternating 
years as 10- to 15-minute design tasks, in which students 
provided written responses to questions concerning the design 
of a retaining wall to contain river flooding (first and junior years) 
and the design of a pedestrian crossing at a busy intersection 
(sophomore and senior years). A third design problem 
concerning the design of a playground took the form of a forced- 
choice survey question administered in the first and senior years. 

These longitudinally collected design task and survey responses 

were analyzed to examine how much students consider various aspects of design problem 
context and the extent to which this changes during the four years of undergraduate 
engineering education. 


First-year students consider design problem context. 

Analyses of first-year students’ responses to the design task (referred to as the “Midwest 
Floods” design task) and the playground design survey question suggest that beginning 
engineering students did take into account the broader context of design problems. 
Examples of broader contextual factors considered include the natural environment, specific 
groups of users or stakeholders, and economic impacts. 

The Midwest Floods task was administered once in the longitudinal cohort’s first year and 
again in their Junior year, with a total of 79 students participating in both years. In this 
design task, students were given 10 minutes to list as many factors as they could think of 
that would be relevant to the design of a retaining wall to contain flooding on the Mississippi 


Figure 2.6-C: Emphasis on various aspects of problem context as reflected in APS 
students’ responses to a problem concerning the design of a retaining wall to contain 
river flooding. Chart shows average results of coding responses of first-year students 
(n = 79), with physical location codes on the horizontal axis and frame of reference 
codes on the vertical axis. Area of the circle for each code pair is proportional to the 
average number of factors matching the code pair. 
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River. Students’ written responses were segmented into distinct ideas (each representing 
one factor), and then the ideas were interpreted and categorized in terms of the physicai 
iocation to which a particular factor referred (e.g., the wall itself or the bank on which it would 
be built) and the frame of reference of the factor (e.g., a technical consideration like the 
height of the waii or the rate at which the river flows would be coded technical; a concern for 
the environmental impact wouid be coded natural). Figure 2.6-C shows how students’ ideas 
were interpreted and categorized along each of these two dimensions, with physicai location 
on the horizontal axis and frame of reference on the vertical axis (Kilgore, Atman, et al., in 
Journal of Engineering Education, 2007; Atman, Kilgore, et al., at Research on Engineering 
Education Symposium, 2008). For instance, the figure shows that first-years’ responded with 
an average of 3.3 wali-related, logisticai factors and 1.6 water-related natural factors. 

To facilitate interpretation, factors referring to the wali itself or the water surrounding it and 
representing a technical or logistical frame of reference were labeled close context factors. 

In the two-dimensional coding diagrams, close context factors are in the lower left corner; 
examples include costs, budget, and technical specifications. In contrast, broad context 
factors referred to the bank or surrounding area, or focused on the naturai/environmental or 
social context issues relevant to the design problem. Examples of broad context factors 
include weather patterns, natural environmental impact, economic impacts, and effects on 
the people, businesses and communities surrounding the proposed retaining wall. The 
coded responses of first-year students, represented in Figure 2.6-C, show roughly equal 
emphasis on broad and close context, in terms of average number of factors of each kind 
considered. 

As with the Midwest Floods design task, first-year student responses on the playground 
design survey question indicate consideration of both the close and broad context of a 
design problem. Students were given a list of 17 kinds of information and asked to choose 
the five that they would most likely need to design a playground. As first-year students, at 
least 80% included Budget and Safety among their choices. Other frequently chosen items 
were related to materials and included their cost, availability, and specifications. As with the 
Midwest Floods design task, these responses indicate consideration of both the ciose and 
broad context of a design problem. (Kilgore, Atman, et al. 2007) 


Many sophomores do not consider design probiems in temporai context. 


Consideration of temporal 
context was analyzed using a 
coding scheme based on life 
cycle analysis. 


The analyses described above examine consideration of physical, 
environmental, societal, and other dimensions of design problem 
context. Extending context to inciude the dimension of time, we 
aiso examined whether students piace design problems in a 
temporal context. Data for these analyses came from the Street 
Crossing design task, administered to 58 Longitudinal Cohort 
students in their sophomore and senior years. Students were 
given 15 minutes to answer a sequence of fourfree-response 
questions about designing a pedestrian crossing at a busy 
intersection. Two of the questions concerned proposal and 
evaluation of potentiai solutions, and responses to these 

questions were coded using a scheme based on life cycle analysis, a technique commonly 
used to assess the sustainability of a designed artifact or process. Our coding scheme 
utilized four stages of a designed solution’s iifetime, listed in chronological order: current 
state, design/construction, solution in piace, and maintenance/disposal. 


When students completed the Street Crossing design task in their sophomore year, most 
considered at least two points in time: the current state (manifesting the probiem to be 
solved) and the solution in place (with the solution implemented as designed). In contrast. 
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only about a quarter of the sophomores considered the design/construction process, and 
less than a fifth thought about the future of the solution by considering its 
maintenance/disposal stage. (Kilgore et al. 2010; Yasuhara et al. 2009) 


2.6.4 Development of Consideration of Context During Design 

With multiple longitudinal data sets examining the consideration of context, we examined 
how two or more years of undergraduate engineering education influenced the ways in which 
students approach design problems. The findings suggest that, while engineering education 
is helping students develop as designers, graduating engineering majors do not exhibit a 
greater appreciation of the importance of the broad context of a design problem. 


Students do not consider broad context more as juniors and seniors than as first-years. 

Comparing the Midwest Floods responses of the Longitudinal Cohort students as first-years 
and Juniors, the Junior-year responses, on average, reflected consideration of more factors 
than the first-year responses. However, this increase was largely limited to consideration of 
close context factors, not broad context. The average number of close context factors 
increased from 6.0 to 8.3 (p < 0.001), but there was no significant increase in the number of 
broad context factors (a = 0.05). (Atman, Kilgore, et al. 2008) 


Graduating engineering majors 
do not exhibit a greater 
appreciation of the importance 
of the broad context of a design 
problem. 


This increased consideration of close context might be a result 
of students’ increased engineering-specific knowledge. This 
growth in engineering-specific knowledge also was reflected in 
students’ shifting conceptions of engineering design 
(Subsection 2.6.1). Also consistent with increased gains in 
engineering-specific knowledge, APS students’ senior-year 
responses to the playground design survey question were more 
similar to those of practicing, professional engineers’ 
responses (data collected as part of another study) than the 
students’ first-year responses were (Atman, Kilgore, and 

McKenna 2008). Finally, initial analyses of the Microchip Factory design task showed that 
very few seniors consider ecological factors when selecting a site for a microchip fabrication 
facility (Atman, Kilgore, et al. 2008). Collectively, these findings suggest that engineering 
curricula could be improved by emphasizing broad context in teaching design. 


Consideration of temporal context does not develop significantly by senior year. 

Comparing Street Crossing design task responses of the Longitudinal Cohort students as 
sophomores and seniors, we observed a similar pattern, with most students failing to 
consider design/construction or maintenance/disposal when proposing and evaluating 
potential solutions. More generally, between the sophomore and senior years, there were no 
statistically significant changes (a = 0.05, n = 58) in the proportions of students considering 
the four life cycle stages we coded for. (Kilgore et al. 2010; Yasuhara et al. 2009) 


2.6.5 Gender Differences in Consideration of Context 

Women are more likely to consider certain aspects of broad context during design. 

Earlier, we observed some differences in the ways in which women and men conceptualized 
engineering design (Subsection 2.6.1). In addition to these differences in how women think 
about design, we observed that women were more likely to consider certain aspects of a 
design problem’s broad context. In the Midwest Floods design task, women responded with 
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more broad context factors than men in both their first and junior years (p < 0.05). (Notably, 
women and men responded with about the same number of close context factors.) (Atman, 
Kilgore, et al. 2008) 

There were similar gender differences in first-year responses to the playground design survey 
question. When asked to select the kinds of information they would most likely need to 
design a playground, women were more likely to select items associated with broad context: 
Neighborhood demographics, Handicapped accessibiiity, and Utiiities (p < 0.05). Conversely, 
they were less likely to select items associated with close context: Budget, Materiai costs, 
and Labor avaiiabiiity and cost (p < 0.05). (Note that, unlike in the Midwest Floods design 
task, which consisted of a free-response question, the fixed number of selections in the 
playground design survey question enforced a zero-sum game in selecting broad vs. close 
context items.) (Kilgore, Atman, et ai. 2007) 

Findings from the Midwest floods design task and the playground design survey question 
suggest that the women gave more attention than men did to the respective design 
problems’ geographical, natural, and/or societal context. When it came to the inclusion of 
temporal context in the Street Crossing design task, however, there were no significant 
gender differences. Women and men were comparably likely (or unlikely) to consider each of 
the coded life cycle stages. These design task and survey findings suggest that women and 
men can take different approaches to engineering design, at least with respect to certain 
aspects of problem context. 
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'in Looking Beyond Graduation: Student Plans 

Although the Academic Pathways Study focused primarily on students’ undergraduate years, 
several components of the research examined seniors’ post-graduation plans. This section 
describes aspects of students’ plans upon graduation. We also consider how these vary by 
gender and underrepresented racial/ethnic minority (URM) status, and what other factors 
come into play in students thinking about their next steps. 

2.7.1 What Do Students’ Post-Graduation Plans Look Like? 

One aim of the APPLES study was to explore engineering students’ post-graduation plans— 
how many students are planning on engineering work, how many are thinking about non- 
engineering graduate school, whether they are considering multiple options, what factors 
help indicate a student’s direction, etc. 

Key findings are summarized below, drawing from descriptive statistics on students’ post- 
graduation plans. Four regression models of senior-level, post-graduation engineering and 
non-engineering school and work plans allowed us to identify and explore relationships 
between motivation, college experiences, demographics, and these plans. Two models of 
first-year student plans towards engineering graduate school and/or engineering work were 
also built. The regression models are described in detail by Sheppard et al. (2010). 

Figure 2.7-A shows that women’s and men’s post-graduation engineering work and 
engineering graduate school plans are similar. 


Figure 2.7-A: Intentions of senior men and women to study and practice engineering 
post-graduation (Gender differences are not statistically significant.) 



Women, Men, Women, Men, 

Work in engineering job Work in engineering job Attend engineering Attend engineering 

grad uate school grad uate school 


■ Definitely/probably □ Unsure □ Definitely not/probably not 
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Nearly 80% said “yes’ to engineering work, and 20% were unsure or leaning away. 

That a high percentage of seniors reported plans to enter into 
the work that their degree aims to prepare them for is 
consistent with what we saw with the Longitudinal Cohort, 
where the commitment to practicing engineering after 
graduation for those who persisted in an engineering major 
increased over their college years (Eri§ et al. 2010). 


Twenty percent of seniors do not 
plan to pursue engineering 
options after graduation or are 
unsure. 


At the same time, it seems surprising that 20% of seniors 
intending to finish an engineering degree were either turning 
away from a future in engineering or remained unsure. It is 
possible that some of them may have long had plans that 
Included engineering as a stepping stone Into another field, 
such as medicine, law, or business. It is also possible that 

some have been “turned off” by engineering along the way but decided to finish anyway (also 
see Subsection 2.3.1 and discussion on doggedness), or were not able to obtain an 
engineering Job offer that they were excited about. Additionally, students who Indicated they 
were “unsure” two months before graduation might have been waiting for Job offers to 
become finalized. 


That a significant number of engineering graduates were leaning away from engineering work 
was also seen in our Longitudinal Cohort studies. These data also indicated that students 
completing a major in engineering were not necessarily committed to careers In engineering. 
In an analysis of data from the senior year (spring) administration of the PIE survey at two of 
the campuses, approximately 26% were either Probably or Definitely Not planning to pursue 
careers in engineering (Lichtenstein et al. 2009). 


Co-ops, internships influence post-graduation plans to pursue engineering. 

The regression models indicate one particular college experience that may influence 
students’ Job direction. Among seniors, the top predictor of engineeringjob plans was 
exposure to engineering experiences such as co-ops and internships. This same variable 
was a negative predictor for plans for non-engineering employment. While those who are 
disinclined towards engineering employment may avoid exposure to co-op and Internship 
experiences, we should also consider the possibility that their employment decision-making 
might have been influenced. If these students had had those experiences. 


Forty percent considering engineering graduate school 

The percentage of students planning on engineering graduate school remained constant at 
over 40% between first-years and seniors. In contrast, a significant number of students 
seem to have ruled out engineering graduate school between first year and senior year. The 
percentage of students not planning on engineering graduate school Increased from 19% in 
their first year to 31% in their senior year. 

We found that the top predictors among seniors of engineering graduate school plans were 
GPA and psychological motivation (e.g., thinking engineering is fun), and the top negative 
predictor was confidence in professional and Interpersonal skills (e.g., skills In written and 
oral communications, teamwork, leadership). That is, students with greater confidence in 
professional and interpersonal skills were less likely to Indicate Interest in graduate school 
(See Subsection 2.7.3). 

Finally, undergraduate research experience only weakly predicted engineering graduate 
school plans, and there was no predictive power to exposure to the profession, academic 
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involvement, or frequency of interaction with instructors. Some students, by the time they 
were seniors, might have developed a more realistic view of themselves in relationship to 
engineering graduate school and their probability of admission and success. Additionally, it 
could be expected that many students were excited to leave school, enter into the 
engineering work world, and begin to make an income, rather than continue studies as 
graduate students. (Note that this data was collected prior to the economic downturn that 
began in 2008.) 

Seniors still unsure about their plans 

While only 8% of seniors were unsure about their plans to enter into engineering work, 
approximately one-quarter were unsure about their plans related to a non-engineering job or 
graduate school (either engineering or non-engineering). In other words, one in four seniors 
was considering how non-engineering options might fit into their future. This is perhaps to be 
expected of students who were entering a period of exploration that frequently occurs 
between adolescence and adulthood. 

In thinking about these results, we also need to recognize that current students (the so- 
called Generation Net or the Millennials) engage differently not only in their education when 
compared to prior generations (Chubin et al. 2008) but also with respect to their futures. As 
seniors, many are still figuring out their interests, what Job opportunities are out there, and 
what new opportunities might emerge. Some are graduating from an engineering program 
and have little idea about the kind of work they will do on a daily basis, despite having 
completed internships or co-ops. This uncertainty about what is in store for them might also 
make students hesitant about engineering work or graduate school (Jocuns 2008). 

Longitudinal Cohort analyses also reflected uncertainty in seniors about their post-graduate 
engineering plans. In addition, they suggest that the APPLE survey findings on students’ 
plans as reported in late winter through early spring of their senior year may have over- 
reported the level of certainty in these plans. Of 28 Longitudinal Cohort seniors we 
interviewed at two of the study schools, 15 fell into the Unsure category. These students 
were still vacillating between different post-graduate options late into the senior year, even 
into summer. To be clear, these students were not vacillating between different job options 
within engineering; they were wrestling with a range of career choices. As one student said. 

Ten years from now, I still plan on being in business for myself, doing consulting, 
either in education, or, possibly, in engineering. But the focus will be on education, 
science education stuff. I might have to do the engineering to, you know, actually 
make some money, but the focus is hopefully going to be in science education. 
(Lichtenstein et al. 2009) 

More than 60% of engineering graduates had a combination of plans. 

As seen in Figure 2.7-B, about one-third of seniors saw themselves as focused on 
“engineering only.” Upwards of 60% of seniors were considering some combination of 
engineering and non-engineeringjobs and/or graduate school. This may have been because 
some were still defining their path and were leaving options open, whereas some might have 
had a defined path with different segments that incorporate work inside and outside of 
engineering. 

We note that seniors, as a group, relative to first year students, were broadening their career 
interests. This may reflect today’s professional reality— students may perceive that they no 
longer have the option of setting their sights on one thing and one thing only. Even though 
students’ career trajectories were becoming more defined by their senior year, there was still 
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Figure 2.7-B: Combinations of options being considered by seniors. (Note: Students 
were able to select from four, non-mutually exclusive options: the likelihood of 
pursuing engineering work, non-engineering work, engineering graduate school, and 
non-engineering work.) 


Four options 
23% 


Engineering focus 
(work and/or 
graduate school) 
30% 



Non-engineering 
focus (work and/or 
graduate school) 
7% 


Job focus (engr. 
and/or non-engr.) 
4% 


Three options 
35% 


Graduate school 
focus (engr. and/or 
non-engr.) 

1 % 


a fair degree of uncertainty as to their paths. Alternateiy, we couid interpret this uncertainty 
as a result of the fact that students’ career trajectories were stili being formed. 

An engineering degree can provide a basis for many future options. 

Our Longitudinal Cohort studies reinforce that many students do not think of their future in 
terms of a single Job or even one career, but rather as one that consists of many possibilities 
and includes non-engineering endeavors. Participants feit that their engineering education 
and the problem-soiving skilis they had learned would provide a good basis for exploring 
different options after graduation (Lichtenstein et al. 2009). 

2.7.2 The Demographics of Tomorrow’s Professionals 

URM students were initialiy more interested in engineering graduate schooi. 

Engineering graduate school plans differed notably between URM (underrepresented 
racial/ethnic minority) and non-URM students. Particularly in the first year, URM students 
expressed significantly more interest in attending engineering graduate school than did non- 
URM students (65% vs. 38%), and URM status was a predictor of engineering graduate 
school plans among first-year students. 

By the senior year, URM status was no longer a predictor, when controlling for other factors. 
However, when we look at the actual percentages, one-and-a-half as many URM seniors 
expressed plans to attend engineering graduate school (Figure 2.7-0). One might expect that 
this stronger interest in engineering graduate school among seniors would translate into 
actual graduate school enrollment, but a gap remains between URM student representation 
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Figure 2.7-C: Seniors in the APPLES sample who are considering engineering 
graduate school, by gender and URM status 



Women Men 

OURM nnon-URM 


in engineering graduate programs reiative to their representation in bachelor’s degree 
programs. 

URM women and men think differently about post-graduation options. 

We note two other observations with respect to student diversity. First, senior URM women 
reported the highest mean level of professional and interpersonal confidence and the lowest 
mean self-reported GPA, relative to their senior peers. In the model of seniors’ engineering 
graduate school plans, GPA was a positive predictor, and confidence in professional/ 
interpersonal skills was a negative predictor. It could be that we are losing some women 
before engineering graduate school because they believe that a lower GPA precludes 
graduate studies. Alternatively, they might not see how their professional and interpersonal 
skills can be utilized in engineering graduate work. 

A second issue is that more senior URM students were considering multiple options that 
span engineering and non-engineering than were non-URM students (67% vs. 56%). This 
could suggest that URM students may have broader interests, and also that engineering as a 
profession may need to work harder to retain these individuals among its ranks. 

Women’s plans similar to men's, but... 

Similar percentages of women and men were planning on engineering and non-engineering 
work and graduate school. However, men were slightly more likely to focus on engineering 
only, and women slightly more on both engineering and non-engineering options. 

2.7.3 Key Factors in Plans 

We saw a mix of factors in looking at students’ plans for the future. Three of the most 
noteworthy were students’ psychological motivation, the level of confidence they expressed 
in their professional/interpersonal skills, and aspects of the institutional setting. 
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Psychological Motivation/ 
Interest strongly predicts 
seniors’ intention to pursue 
engineering options after 
graduation 


Psychological motivation/interest an important factor 

The psychological motivation factor loaded heavily on all four of 
our senior regression models. It positively predicted heading 
toward post-graduate engineering options and negatively 
predicted heading toward non-engineering options. This makes 
sense in that individuals who are excited by engineering want to 
keep doing engineering. However, one question that we did not 
investigate is, where does psychological interest and motivation 
come from? Thinking of the experiences of students beyond the 
campus and even before they enroll in college, could K-12 education, families and 
communities, and society do more to promote enjoyment in engineering thinking, through 
more exposure, more opportunities, and more support in general? Additionally, engineering 
might be conceptualized in ways that invite a more diverse student population by highlighting 
the broad range of problems that engineers address. 


Confidence in professional and interpersonal skills an important factor 


Confidence in professional and interpersonal skills also showed 
up as being important in all senior models, but in the opposite 
direction to that of psychological motivation. We found that 
heading towards engineering options was predicted by lower 
professional/interpersonal confidence (all other factors being 
equal), whereas heading towards non-engineering options was 
predicted by higher professional/interpersonal confidence. 
However, employers increasingly say that these skills are critical 
for success in today’s engineering workplace, as the practice of 
engineering is a social, as well as technical, activity. 


Confidence in Professional & 
Interpersonal skills strongly 
predicts seniors’ intention to 
pursue non-engineering options 
after graduation 


Institutional differences can have strong influences on student pathways. 

The responses of engineering seniors at two of the four APS campuses from the Longitudinal 
Cohort were examined in greater detail with respect to their thoughts and intentions to follow 
an engineering pathway after graduation. The PIE survey and semi-structured interview data 
from 74 students on the two campuses were analyzed, and four major themes emerged that 
create a richer portrait that adds to the cross-sectional APPLE survey results of student 
intentions to pursue (or not) an engineering career after graduation. 

The seniors on the two campuses represented in this subset of APS data showed different 
response patterns when asked about whether they planned to pursue careers in engineering. 
These different response patterns were likely a function of programmatic differences at the 
two campuses. One campus was focused primarily on the education of engineering, science, 
and technology majors, and the other larger university offered a broader range of majors, 
including the humanities and social sciences. At the technical campus, only 14% of seniors 
indicated they were unlikely to pursue an engineering related activity after graduation (either 
a job or graduate school). On the other campus, 36% of engineering seniors reported that 
they were unlikely to pursue an engineering related activity. These differences emphasize 
the potential for different institutions to attract different kinds of students and/or for 
institutional factors to shape post-graduation plans. (Lichtenstein et al. 2009). 
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2.8 Looking Beyond Graduation: Experiences in the Work World 

As the previous subsection described, approximately 80% of engineering undergraduates in 
our sample were planning to seek employment in an engineering field. Although the APS 
focused primarily on students’ undergraduate years, several components of the research 
examined the experiences of recent graduates’ entry into the engineering workforce. 

In a series of studies during 2006-2009, we interviewed over 100 engineering graduates 
who were in the first or second year of an engineering job and 15 of their managers. 

Fourteen companies were represented, and the studies generated six distinct data sets. 

The major themes that emerged about the engineering experiences of early career engineers 
are presented below. This work has been situated in multiple theoretical frames looking at 
the transition from school to work— specifically, the ways in which new engineers make sense 
of their career choices and learn to navigate their first job experiences. Theoretical frames 
used for analysis were Social Cognitive Career Theory (Lent and Brown 2008), Social 
Cognitive Theory (Bandura 1986, 2001), Social Exchange Theory (Blau 1986, Lawler 2001), 
and theories of relationships (Villard and Whipple 1976). 

In these studies of new engineers entering the workplace, there were two predominant topics 
studied: the experiences of new engineers applying their technical expertise (what they 
learned In school) to engineering problems, and their experiences learning to navigate the 
organizational and social systems of the workplace. 


2.8.1 The Work World, as Compared to the School World: 
The Nature of Engineering Problems 


Technical problems are more complex and ambiguous in the work world. 


Work problems were ambiguous 
and more complex and often 
lacked complete data. 


Some newly hired engineers remarked that the technical 
problems they faced on the job were different from those they 
encountered in school. Work problems were ambiguous and 
more complex and often lacked complete data. Frequently, a 
necessary Initial task was to define the problem In terms of the 
preferences of the group, thereby deciding on the appropriate 
scope and what information was key to the solution. It was 
Important to understand the larger work group’s expectations 
about what constituted an appropriate approach and an 
acceptable solution. New engineers consistently reported that 
the work of engineering they encountered on their first jobs 
was significantly different from their experiences in school. 

(Korte, Sheppard, and Jordan, 2008) New hires reported encountering a steep learning 
curve, initially, and this was sometimes associated with a probationary period. Recent hires 
also reported a change In their sense of time, in that problems could pop up suddenly and 
that deadlines were less under their control than when they were In school. (Jocuns and 
Stevens 2009) 


For example, talking about the differences between work and school problems, one of the 
new engineers said. 
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I mean, in school, it’s very textbook. They always try and model everything in a 
mathematical sense in school. And in the real world, it’s a lot more difficult to model 
things. It’s just there’s a lot more variables involved, and there’s the [uncertainty] 
too of whether or not you’re modeling it right. Are you following the right procedures 
and principles? And stuff like that. (Korte, Sheppard, and Jordan, 2008) 

Another new hire had a similar feeling, saying, 

/ guess in school, you’ve got your specific material you’re learning, and you have your 
specific problem that goes with that material. In real life, you never have anything 
that specific, at least with my experience so far. I’m sure there are some people that 
do, but then they’ll only have those kinds of specific problems. Mine, I have to 
combine that with 500 other specific problems that we might have done in school. 

It’s never Just that one focus. (Korte, Sheppard, and Jordan 2008) 

Another new engineer described this difference between school and the work world: 

Whereas in real life, right, you’re not going to just bend the thing, see what happens, 
and explain it, and then move on to your next assignment. You’re going to bend that 
thing, see what happens, explain it, and try and fix it. Whereas in school, you explain 
it and get a result, and then no follow-up. (Korte, Sheppard, and Jordan 2008) 

We note that the type of assignment given to a new hire could determine how strongly linked 
they felt the activity was to their schooling. Tasks that involved problem definition and 
resolution were felt to be more strongly linked to what they had learned in school. Many 
referred to learning a “way of thinking” that typified engineering expertise. Other tasks that 
were focused on project or data management seemed less relevant to their education and 
less like “real engineering.” 

Many different players and processes can affect decisions. 

New hires found that there were many people who could 
Influence decisions, including those who might not be directly 
Involved with a team. Data that had been regarded as objective 
and the primary basis for decisions in school might not, in the 
workplace, have this objective status or the same top priority in 
reaching a decision. In many cases, priorities were based on the 
relationships between the engineer and others in the 
organization. In addition, company processes were sometimes 
perceived as a constraint to effectiveness or efficiency. New 
hires needed to learn skills to navigate the more nuanced 
decision-making process in their place of employment. (Korte, 

Sheppard, and Jordan 2008) 


Company processes were 
sometimes perceived as a 
constraint to effectiveness or 
efficiency. 


2.8.2 The Work World, as Compared to the School World: 

Managers, Co-Workers and Teamwork 

Support from managers and coworkers is very important and can vary greatly. 

There was great variability in the degree to which the new hire’s manager provided direct 
assistance. For some, their manager was actively involved in providing information, 
answering questions, or pointing new hires towards resources. For others, managers 
provided little assistance, and this was often because they were too busy in balancing project 
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assignments and other managerial duties. In some cases, the manager was not even 
available to greet the newly hired engineer on their first day. Another frequent cause for lack 
of support was that the manager was relatively new to the company or manager’s role and 
therefore still learning the ropes. (Korte 2009) 

The experiences of new hires ranged from good... 

Well, [companyj’s a big organization, and I think I was very fortunate, because, when 
I started the position, my Initial supervisor was very— they were very good at throwing 
things at you that you might not even know why you need them. 

...to less satisfactory: 

My manager wasn’t there to greet me, or nobody was there to be like, “Hey, welcome 
aboard,” blah, blah, blah. There was like one random dude that happened. ..They 
were like, “Do you know where [manager] Is?” “No, I don’t know where he Is.” And 
now I know why they don’t know where he Is, because he’s busy as hell, and he’s 
never at his desk. (Brunhaver et al. 2010) 

Compared to managers, co-workers were the most significant source of information in this 
process, both in terms of understanding work tasks and the culture of the work group. New 
hires were often given assignments that had been formulated by more senior engineers. 

New hires often had to build good relationships in the group to access sources of technical 
Information, as well as for learning the context and social systems of the workplace. (Korte, 
Sheppard, and Jordan 2008; Korte 2009, 2010) 

As one study participant reported. 

It’s like, around here, you’re going to run Into a lot of people that are very laid back, 
and If they don’t think you’re priority or your work’s priority, you’ll be on the back 
burner for a year on something. And so you’ll learn that you’ve really got to network 
and really learn people around here and really, really get to know them on a personal 
level and earn their respect. (Korte, Sheppard, and Jordan 2008) 

The assistance new hires were able to get from their coworkers also varied greatly. Similar to 
the situation with managers, some coworkers were very supportive in mentoring the new hire 
and orienting them to the job. In other cases, coworkers were too busy or new at the Job 
themselves to offer much assistance. (Brunhaver et al. 2010, Korte 2009) 

Differences in age or outside interests can impede camaraderie. 

With respect to forming relationships with coworkers (“bonding”), there was a broad range of 
experiences for new graduates, from developing a strong feeling of camaraderie with a 
majority of their coworkers to feeling little or no camaraderie. A frequent cause of the latter 
situation was a significant difference in age or outside interests. As one former student said 
of his successful bonding with coworkers. 

It was much easier because I think we were all In the same age group, so Integrating 
with the group was really easy. I think we got to know them, they got to know you. 

And It’s like I was here for maybe ten years, I mean It Just— you Just have that type of 
feeling to It, you know? (Brunhaver et al. 2010) 

Rotation of new engineers can inhibit forming strong reiationships with coworkers. 

The practice of rotating new engineers through different positions as part of orientation was 
a program at several of the companies we studied. This practice can have an impact on how 
well new hires learn the business of the organization and develop relationships in a work 
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group. Rotating around the company afforded new hires experiences with the range of 
activities that comprised the business. Rotations would often last three to six months, and 
new hires sometimes felt that, despite the Importance of learning from coworkers, it was not 
worth the effort for some of their coworkers to befriend a new engineer who was unlikely to 
stay beyond the duration of the rotation. As one new hire who had negative feelings about a 
rotation experience said. 


You’re only going to be there for four months. So you could understand where 
people are, like, well. I'm not going to take the time to really get to know you, ’cause 
you’re going to be gone in three months. And so you don’t really take it personally, 
but you do kind of. (Brunhaver et al. 2010) 


Teamwork was much different in the workplace than in school. 

In many cases, new hires started working on teams well after the 
project had started and sometimes never saw the finish of the 
project. This view of teamwork was different than school 
projects, in which the members typically started and finished a 
project together. (Korte, Sheppard, and Jordan 2008) 


Teamwork can be much more 
complex in the workplace 
compared to school: Joining or 
leaving a project mid-stream; 
working with non-engineers... 


Teams In the workplace were usually much larger and more 
diverse than those that students experienced in school. 

Workplace teams often included members from other engineering disciplines, others who 
were non-engineers, and occasionally people from outside the company. New engineers 
could sometimes be the sole engineering (or company) representative on these teams and, 
as a result, had a greater sense of responsibility. (Jocuns 2009) 


2.8.3 Finding One’s Roie or Piace 
Understanding one’s role 

A theme running through many of the interviews was the importance to new hires of under- 
standing what their role was in the company. This could be a source of concern, since we 
heard only a few examples in our interviews of managers who made the effort to provide this 
sort of context to their new employees. Engineering work was often a complex, long-term 
project. Individual engineers contributed small pieces of the project, and it was difficult for 
new hires to understand how the whole process worked. (Korte, Sheppard, and Jordan 
2008) 

Getting a sense of the bigger picture 

A key part of understanding one’s role was learning the reasoning behind decisions, as well 
as the goals and priorities of the larger organization. Understanding the priorities of the 
organization allowed new hires to formulate ideas about the value of their work and how it fit 
into the larger picture. (Korte, Sheppard, and Jordan 2008) 

Some new hires expressed feeling a bit lost navigating in a large organization. In the words 
of one, “I had a hard time Just trying to understand how the system actually worked. Even 
now. I’m still a little iffy on a lot of details and whatnot.” Another stated, “One of the first 
things is kind of like to get oriented into the actual whole system, because the system is Just 
like absolutely, ridiculously huge.” (Korte, Sheppard, and Jordan 2008) 
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Company education efforts could be insufficient 

All companies offered some type of education for the new 
employee to varying degrees, and in some cases this effort was 
quite extensive. In spite of this, new engineers typically indicated 
that they felt their company’s efforts in training and new 
employee orientation did not seem sufficient. This often was 
expressed as a lack of exposure to “the big picture" and an 
incomplete (or non-existent) description of where the individual 
fit in the corporate structure. 

As one study participant who was hungry for a view of the “bigger picture” said, 

/ wanted always more overview, more overview. Tell me about how the whole 
company process and procedures work. How does this person fit in with that 
person? I was getting into too much depth of information on specifics without 
getting an overview. So I was constantly asking people, especially my first two, three 
weeks, give me an overview. (Brunhaver et al. 2010) 


New hires often felt the lack of 
exposure to the “big picture” and 
were unsure of where they fit 
into the organization. 


2.8.4 Communication Matters 

The importance of communication and documentation 

New hires reported becoming aware of the importance of effective communication and 
documentation in the workplace. This could be routine reporting to a manager and the team, 
as well as sharing results with other interested groups in the company who might not have 
been directly involved in their project. In some cases, new hires began projects that had little 
documentation or background material and had to fill in the gaps for themselves. Finding 
information was a major task for new hires and included finding out who knew what, what 
information existed, and where this information was. Navigating myriad databases and 
learning who to ask for what information was important to getting grounded in the job. 

(Korte, Sheppard, and Jordan 2008) 

One new engineer felt the challenge of having to pick up the pieces from someone else, 
stating, “The person in charge of this system, he already had a test procedure that he had 
started writing, but he had written that, and it wasn’t really complete, like two, three years 
ago. So it’s kind of like he handed that over to me, and I had to update it and make changes 
and kind of get it up to date with the new spec that was in the system now.” (Korte, 
Sheppard, and Jordan 2008) 

Communicating with non-engineers 

One significant aspect of the job for many newcomers was the 
importance of communicating with non-engineers. As part of 
this, new hires had to learn to anticipate how different interests 
in the organization might interpret data in different ways, and 
this was a new experience for many (Korte, Sheppard, and 
Jordan 2008). An important and necessary skill was learning 
how to communicate effectively with audiences that included 
clients and others who were not engineers (Jocuns 2009). 

One participant described learning the importance of understanding the audience when 
presenting results: “[I am] definitely learning more about how to present my data to people... 


New hires had to learn how to 
communicate effectively with 
audiences that included clients 
and others who were not 
engineers. 
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It’s a huge difference in how people perceive your data depending on how much they know” 
(Korte, Sheppard, and Jordan 2008). 

Learning to use a new language 

Many new hires had to learn both company- and industry-specific language. In addition to 
learning entirely new terminology, new hires had to learn ways of referring to things that were 
different from vocabulary used in school. Mastering this new language required learning how 
to use the terminology in different contexts. (Jocuns 2009) This acquisition of terminology 
can be seen as a continuation of learning the language of engineering that began during the 
undergraduate years, as discussed in Subsection 2.5.1. 
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2.9 Summarizing Results about Diversity 

The percentage of engineering degrees received by women and underrepresented minorities 
(URM) has remained markediy flat for the past decade— about 20% for women and 5% and 
7% for African American and Hispanic students, respectively (NSF 2009). An important goal 
of the Academic Pathways Study was to assist efforts to increase diversity in engineering 
education by providing greater insight into the variety of experiences diverse engineering 
students may have. Therefore, we oversampied for gender and race/ethnicity in our 
longitudinal and cross-sectional studies to ensure the inciusion of underrepresented voices. 
Having sufficient numbers of women and URM (underrepresented raciai/ethnic minority) 
students enabied us to examine how gender and URM status are reiated to the various 
student trajectories we observed. 

In the prior subsections, we described learning, developmental, and professional pathways of 
engineering students as indicated by APS findings. In this subsection, we summarize 
findings with respect to gender and URM status from across the other subsections. We 
follow the order of previous subsections, discussing findings about engineering students’ 
overall college experience: motivation and interest in engineering; experiences within the 
engineering college; development of engineering and design knowledge, conceptions, and 
confidence; and post-graduation plans. 

2.9.1 The College Experience 

As discussed in Subsection 2.2, the engineering major is less diverse than other majors with 
respect to gender, URM status, and first-generation coiiege students. Within the NSSE 
Comparative, Longitudinai data set (247 institutions, 11,812 participants), we found that a 
larger proportion of in-migrators than persisters in engineering were women, URM students, 
and first-generation coliege students. (Recall that in-migrators are students who start college 
in a non-engineering major but subsequently switch into engineering. Persisters are students 
who start and stay in engineering.) In other words, the group of students who migrated into 
engineering was a source of increased diversity. Unfortunately, engineering does not appear 
to invite in-migration, relative to other coiiege majors. Only 7% of engineering persisters in 
their eighth coiiege semester came from majors outside engineering, compared with other 
majors in which in-migrators comprised 30-65% of students at this stage (Ohiand et al. 
2008). Below, we discuss some of the areas where differences in perceptions, perspectives, 
and experiences may provide some explanations for continuing underrepresentation, as well 
as suggesting opportunities for addressing this issue. 

2.9.2 Motivation to Study Engineering 

We recall from Subsection 2.3 that we found psychological motivation (the enjoyment of 
engineering inherent in the activity) and behavioral motivation (related to the practical, 
hands-on aspects of engineering) were the strongest sources of motivation to study 
engineering, with motivation to do social good and financial motivation not far behind. Ali of 
the above sources of motivation were very strong in comparison to mentor or parentai 
influence, but, nonetheless, these additional sources of motivation were also important to 
some of the students in our sampie. 
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Gender, motivation, and approaches to engineering 

Some motivators have different significance for men and women. In the Broader Nationai 
Sampie, behavioral motivation was greater among men than women, whereas mentor 
motivation was greater among women. The greater behaviorai motivation of men is 
consistent with how they prioritized engineering design activities, with men significantly more 
likely to have included “building" among those activities they consider most important in 
engineering design (Subsection 2.6). The perceived importance of the practical, hands-on 
aspects of engineering work was refiected in many students’ comments in interviews. For 
exampie, Austin, a mechanical engineering major at the large, public university, described his 
motivation to study engineering: “I’ve always been interested in, I don’t know if— not 
mechanicai systems, but Just like putting stuff together.” He went on to describe his work as 
an engineering student: “Basicaliy all the design work we do is to assemble something so, 
yeah, it’s aiways— the more stuff you’ve built the better, and I think— I don’t know. Just a lot of 
what I’ve done has paid off that way.” (Kilgore, unpublished) 

In contrast, Lauren, a chemical engineering major at the iarge, public university, described 
men’s and women’s different approaches to project work: 

When we’re working on projects and stuff, [men] have like a one-track mind where 
it’s like, let’s just get through this and then we can go.. .When I’m in a group, then I 
sort of have to pay attention to the little details surrounding it, like, oh, what about 
this, what about this, and maybe we have done this— maybe not get through 
everything in one sitting as they would like, but then consider more of the big picture 
sometimes. (Chachra et at. 2008) 

Gender, motivation, and major 

Students’ majors also seemed to factor into differences in motivation in the Broader National 
Sample. For senior women majoring in mechanical engineering (ME), eiectrical engineering 
(EE), or aerospace engineering (AE), psychologicai motivation, behavioral motivation, and 
social good motivation were of comparabie (and high) strength (Parikh et al. 2009). In 
contrast, for men in these same three fieids, there was a ciear hierarchy: behavioral, 
followed by psychological, followed by social good. Both men and women majoring in 
bioscience-related engineering fieids (BioX) exhibited comparable (high) levels of 
psychological, behavioral, and social good motivation, followed by financial motivation— the 
same pattern as for women in ME, EE, and AE. Nationaliy, BioX enrolis a significantly higher 
percentage of women than ME, EE, or AE, so the motivation profiles observed in BioX might 
reflect the critical mass of women in these majors. Alternatively, students with this 
motivationai profiie might be differentialiy attracted to BioX majors because of content or 
values reflected in the BioX fields. 

Women majoring in industrial engineering (IE) and chemical engineering (ChemE) had a 
distinctiy different pattern than women in other engineering majors. For these women, 
psychologicai, sociai good, and financial motivation were of comparabie (and high) strength, 
foliowed by behaviorai motivation. These women were more financiaiiy motivated to study 
engineering than their AE and BioX counterparts, and iess behavioraliy motivated than their 
AEand EE counterparts. Again, in the case of IE and ChemE, certain characteristics of the 
major may reflect the greater number of women involved in it or, aiternativeiy, may attract a 
greater number of women, or some combination of the two. 

Gender, URM, and mentor influence 

In both the Longitudinal Cohort and Broader National Sample, women reported that mentor 
influence was a greater factor in their decision to study engineering than did men. Although 
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mentor influence was not the strongest factor for engineering students as a whole, it did 
figure prominently for some women. For example, Elizabeth, a computer engineering student 
at the large, public university, was offered continued support by a company hoping to hire her 
upon graduation. In addition to matching her with a mentor when she was interning for 
them, mentors continued to visit her when she returned to school. Elizabeth described 
having periodic conversations over coffee with female mentors who offered practical career 
advice as well as social support. We also recall from the NSSE data analysis that women 
represented a higher percentage of in-mIgrators than persisters in our sample. It may be 
that mentors serve an important role In encouraging diverse college students who are still 
searching for the right major to consider engineering. 

Indeed, a few URM students at one APS Institution were drawn to study engineering because 
they were given a high-paying summer internship in engineering before their first year of 
college, and were offered additional advising and extra tutorials for prerequisite coursework. 
Before their internships, these students had not considered engineering as a potential 
career. Women learned from the experience that one could do many different kinds of work 
with an engineering degree, and not all of them planned to become practicing professional 
engineers. Some were going to use their engineering degrees to do non-engineering work. 
(Garrison, forthcoming) 

2.9.3 The Engineering Coiiege Experience 

As discussed in Subsection 2.4, many of the college experiences of women and men 
majoring in engineering were similar, as reflected in the Broader National Sample. They had 
similar levels of Interaction and satisfaction with instructors, and reported similar levels of 
academic involvement, as well as exposure to engineering through co-ops, internships, and 
research. They also reported similar GPAs. However, we have noted some differences in 
their experiences. First, women and men exhibited differences in levels of participation In 
extracurricular activities, with women reporting higher participation. Women also reported 
greater feelings of curricular overload and difficulty balancing school and social life. 

Gender and extracurricular activities 

It appears that activities outside the classroom may play a more important role in the lives of 
undergraduate engineering women than of men (Chachra et al. 2009). As in the Broader 
National Sample, women in the Longitudinal Cohort reported more frequent involvement in 
engineering and non-engineering extracurricular activities than did men, and in the case of 
non-engineering activities, attributed more importance to these activities. We also saw an 
Interesting gendered pattern of participation emerging in a small sample of graduating 
seniors (n = 15) reflecting on their extracurricular participation in their senior year. Women 
in this sample described their participation in professional and networking organizations, 
while men were more often engaged in hands-on, engineering-related extracurricular 
activities. Also, women more often described taking on administrative leadership roles. 
Finally, among seniors who reported high levels of participation in extracurricular activities, 
women more often described activities in more than one organization, while men more often 
were focused on one intensive activity. 

Gender and curricular overload 

Both In the first and senior years, women in the Broader National Sample reported a greater 
sense of curricular overload (difficulty In coping with the pace and load demands of 
engineering-related courses) than did men. In addition, they reported greater pressure to 
balance their social and academic lives. This sense of overload is unlikely to be a direct 
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result of greater participation in extracurricular activities, as we noted that, for both women 
and men, there was no simple correlation between curricular overload and extracurricular 
activities. 

Gender and race/ethnicity in the classroom 

Elizabeth, the computer engineering major in her last year at the large, public university, 
observed that her contributions to in-class discussions were ignored in favor of men’s: 

Let’s say.. .there are three people in the group and there’s one guy and, you know, 
one girl talking to another person, and then let’s say I’m the girl and there’s another 
guy trying to explain the same thing to another person. That person turns to listen to 
the guy rather than the girl. That happened to me a lot... (Chachra et al. 2008) 

Perceptions like this may have led some women at this institution to believe that they would 
have to prove themselves to the men in their classes before their ideas were considered as 
valuable as those of their male classmates (Garrison et al., Cultural models, 2007). Such a 
perception also goes hand in hand with a sense of the importance of representing their 
gender positively. Some women at the comprehensive, historically Black, private university 
spoke of the importance of doing well to favorably represent their gender (Fleming et al. 
2008). Similarly, at the large, public university, both women and men majoring in 
engineering said in interviews that they believed women were held to lower standards for 
admission to the college than men (Garrison et al., Cultural models, 2007). One male 
student was rather blunt, musing, “I always kind of wonder like, hmm, when girls get 
something, I wonder if it’s because they’re a girl or because they deserved it” (Chachra and 
Kilgore 2009). Among Longitudinal Cohort students who participated in the structured 
interviews, white women in particular seemed to hold this opinion, describing the novelty and 
advantages of being in the minority for the first time, perceiving that they had more support 
and opportunities available to them (Fleming et al. 2008). 

At the large, public university, after admission to engineering (typically occurring at the end of 
the sophomore year), women reported reluctance to seek help on work from men in their 
classes to avoid being perceived as not having earned a spot in the college. Women felt the 
need to maintain the appearance that they knew “what was going on,” so they went 
“underground” for help and tried to work with other women whenever they could (Garrison, 
Stevens, and Jocuns 2008). Some also reported wanting to stay in engineering despite 
negative feelings about the major, because they wanted to make things better for those who 
would follow them (Garrison etal., Cultural models, 2007). 

This concern about others’ perceptions was shared by some URM students, though only 19% 
of students in the Longitudinal Cohort said race plays a part in their becoming an engineer. 
William, an African-American student at the comprehensive, private university, explained his 
own complex perspective: “I feel like sometimes African-Americans.. .aren’t.. .considered to 
be as.. .capable. ..as far as doing course work and everything it takes to become an engineer.” 
William appeared not to have internalized the external negative messages that he perceived 
as he went on to distinguish this external issue from his own beliefs: “I don’t personally feel 
like that’s the truth...” (Fleming et al. 2008). This stands in contrast to Kara, a white student 
who diminished the importance of an award for best female engineer that she received from 
the Society of Women Engineers, saying, “There’s probably only 20 girls in my class” 

(Chachra et al. 2008). These findings appear consistent with those concerning relationships 
between gender and URM status with self-confidence in math and science skills, as 
elaborated below. 
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2.9.4 Engineering Knowiedge, Conceptions, and Confidence 

Women and men reported similar gains in learning about engineering and cited similar 
sources of these gains, as we discussed in Subsection 2.5. We see two noteworthy gender 
differences in the Broader National Sample. First, women reported professional and 
interpersonal skills as being more important for engineers than did men. Second, more 
dimensions of the college experience (interactions with instructors, and “out-of-classroom” 
experiences) correlated with gains in engineering knowledge for men than for women. At the 
same time, women in the Broader National Sample reported lower confidence in math and 
science skills than did men. 

Gender and professional and interpersonal skills 

In the Broader National Sample, first-year and senior women reported professional and 
interpersonal skills as being more important than did men (and this gender difference was 
greater among seniors than among first-years). This is surprising, considering that with 
increasing time in college, students are more likely to have been involved in engineering- 
related summer jobs, internships, and co-ops, as well as team-based design projects where 
these skills would be used. Consider Kara’s reflections on her senior design capstone at the 
large, public university, which she described as “90%. ..logistics and the running around and 
the getting people together and making— or Just, you know, keeping people on task and 
together and so forth” (Kilgore, Jocuns, and Atman, School gets in the way, forthcoming). If 
women and men tend to take different kinds of roles in collaborative learning settings, this 
might help explain gender differences in how these settings are experienced. It might be the 
case, for instance, that women take on more of the project management-oriented tasks and 
men take on more of the technical tasks in a team-based design project. 

Gender and engineering knowledge gain 

In the Broader National Sample, men’s reported gains in engineering knowledge correlated 
with frequency of interactions with instructors and “out-of-classroom” experiences, while 
women’s gains in engineering knowledge did not. As discussed previously, at least on one 
APS campus, women’s desire to appear as strong engineering students drove them to seek 
academic assistance from other women, often outside the classroom. This reluctance to 
publicly admit weakness could offer an explanation as to why we saw no correlation between 
an increase in engineering knowledge and women’s interactions with their instructors in the 
larger sample. Also, we observed different patterns of participation in extracurricular 
activities, with women more likely than men to report taking administrative leadership 
positions in student organizations. Although most students acknowledged the importance of 
the professional and interpersonal skills that would have been developed in such roles, 
students rated these kinds of skills secondary to math and science skills. These out-of- 
classroom experiences may not have been as closely connected to the development of 
engineering knowledge (as reported by students) as other experiences that seemed more 
directly linked. 

Confidence in math and science skills 

In the Broader National Sample, confidence in math and science skills was predicted by self- 
reported GPA (for both women and men). We cautiously interpret this as indicating that this 
confidence was grounded in school-measured academic performance. Female gender and 
lower perceived family income were weak predictors of lower self-confidence in math and 
science skills, whereas URM status was not a predictor at all. However, in some cases, 
gender seemed to be a more significant factor than self-reported GPA, as even some women 
with high GPAs expressed self-doubts. In a series of interviews at the public, technically- 
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focused research university, we saw that some women with consistently high grades can still 
doubt their engineering ability and be uncertain about practicing engineering. Despite her 
high GPA, Leslie doubted she had all the skills she needed to be an engineer and grew to 
define her career goal In terms of taking a support rather than primary role. “[I] visualize 
myself not really actually doing the engineering itself. But, being the support to someone 
else who does it.” In Leslie’s case, though confidence did not deter her from attaining an 
engineering degree, it may have been a factor in the formation of her professional Identity. 
(Matusovich etal., Competence In Engineering, 2009) 

In general, engineering is achieving relative success with respect to persistence in the major, 
as discussed in Subsection 2.2. However, Leslie’s case demonstrates one striking conse- 
quence of lower confidence in math and science skills that may be more prominent among 
women and may perpetuate gaps in professional achievement beyond the college years. 

At the same time, our finding that race/ethnicity seemed not to influence confidence in these 
key skills Is noteworthy. We recall that students were more likely to say gender rather than 
race/ethnicity played a role in their development as engineers. We also are reminded of 
William at the comprehensive, private university, who acknowledged that others may 
perceive his race to be a factor In his abilities as an engineer, but he did not internalize this 
message (see Subsection 2.9.3). 

In the Broader National Sample, frequency of interaction with faculty. Involvement In 
research, participation in engineering extracurricular activities, and exposure to engineering 
through co-ops, internships, and work experience all had no predictive power when it came to 
confidence in math and science skills. In addition, in the Longitudinal Cohort, we found no 
changes overall In the self-confidence measures we took over time. However, we did see the 
same gender gap in math and science skills confidence, as well as in confidence in open- 
ended problem-solving (Chachra and Kilgore 2009). What is it about the experiences of 
women that results in their continuing to have lower levels of self-confidence In math and 
science skills throughout the college years? 

2.9.5 Engineering Design Knowiedge, Conceptions, and Confidence 

One explanation for the persistent gender gap in self-confidence may lie in women’s and 
men’s different conceptions of and approaches to engineering design work, as discussed in 
Subsection 2.6. Below, we discuss gender differences in these conceptions of and 
approaches to design, including potential links between these trends and the gender gap in 
self-confidence. 

Gender and conceptions of design 

When asked to identify the six most important design activities from a list of 23, first-year 
women in the Longitudinal Cohort were more likely than their male counterparts to Include 
Seeking information, while less likely to select Building and Prototyping. Gender differences 
In the students’ fourth-year responses to the same question were similar, with women more 
likely to select Goal setting and less likely to select Building. (Chachra et al. 2008) 

We recall that the opportunity to engage In hands-on activities in engineering was a stronger 
motivator for men than for women in the Broader National Sample. This finding is consistent 
with the tendency for men in the Longitudinal Cohort to think of engineering design in terms 
of hands-on activities, with women tending to consider management and planning activities 
as more important than men did. It may be that both women and men perceive engineering 
as a practical, hands-on endeavor, yet the work that is most valued by women is outside this 
dominant image. 
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Gender and confidence in design abiiities 

In the Longitudinal Cohort, men reported higher confidence and course preparation to do 
design work than women in both the second and fourth years. These gender gaps are 
notable in iight of the absence of gender differences in the reported frequency of 
engagement with the design activities in coursework (in either year). Taken together, these 
findings may suggest that there was a gender difference in the quality (if not quantity) of 
design education that students in this sampie received. As discussed earlier in this 
subsection, it may be the case that women and men tend to take different kinds of roies in 
team assignments, similar to what we observed with women in administrative ieadership 
roles in extracurricuiar activities. If indeed men tend to engage in the hands-on (and more 
directiy visibie) activities and women taking the iess visibie pianning and project 
management activities— this couid result in quaiitatively different course experiences for men 
and women. An alternative interpretation of the confidence gaps is that women may hold 
themselves to higher standards than men when they consider their levei of preparation. This 
interpretation is complemented by findings discussed previously in this subsection about 
women in the Longitudinal Cohort being especialiy concerned about representing their 
gender positively. 

Gender and approaches to design 

Above, we observed some differences in the ways in which women and men conceptualized 
engineering design. In addition to these differences in how women think about design, we 
observed that women were more iikeiy than men to consider certain aspects of a design 
probiem’s broader context. When asked to list as many factors as they couid think of that 
would be important in designing a retaining waii system to prevent flooding of the Mississippi 
River, women in the Longitudinai Cohort responded with more broad context factors than 
men in both their first and Junior years (Figure 2.9-A; Atman, Kilgore, et a/., at Research on 
Engineering Education Symposium, 2008). 


Figure 2.9-A: Average number of close and broad context factors in responses to the 
longitudinally administered Midwest floods engineering design task, by gender and 
year (n = 79). In both the first (p < 0.01) and junior year (p < 0.05), women’s 
responses included more broad context factors. 
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There were similar gender differences in at least the first-year responses to another design 
task administered to the Longitudinal Cohort. When asked to select the kinds of information 
they would most likely need to design a playground from a specific list, women were more 
likely than men to select items associated with broad context: Neighborhood demographics, 
Handicapped accessibiiity, and Utiiities. Conversely, they were less likely to select Items 
associated with close context: Budget, Materiai costs, and Labor avaiiabiiity and cost. 
(Kilgore, Atman, et ai.. In Journai of Engineering Education, 2007) 

Findings from the Midwest floods design task and the playground design survey question 
suggest that the women gave more attention than men did to the respective design 
problems’ geographic, natural, and/or societal contexts. When it came to temporal context 
In the Street Crossing design task, however, there were no significant gender differences. 
Women and men were equally likely (or unlikely) to consider each of the life cycle stages that 
responses were coded for (Kilgore et ai. 2010). 

2.9.6 Post-Graduation Plans 

There were two notable trends In post-graduation plans in the Broader National Sample. The 
first was that URM students were more likely to express an Intention to attend engineering 
graduate school. Second, while similar percentages of women and men expressed an 
Intention to get a job, men were more likely to focus on engineering Jobs, while women were 
more open to non-engineering Jobs as well. 

Graduate school 

Notably, engineering graduate school plans differed between URM and non-URM students. 
Among first-year students in the Broader National Sample, URM students expressed 
significantly more interest in attending engineering graduate school than did non-URM 
students (65% vs. 38%), and URM status was also a predictor of engineering graduate school 
plans among first-year students. 

Among seniors, URM status was not a predictor when other factors were controlled, though a 
substantially larger proportion of URM seniors expressed plans to attend engineering 
graduate school, relative to non-URM students. What is feeding these differences in 
students’ interest in engineering graduate school? How is the higher professional and 
Interpersonal confidence and lower self-reported GPA of senior URM women coming into 
play? And why does the increased interest in engineering graduate school among seniors 
not translate Into actual graduate school enrollment, as a gap remains between URM 
student representation in engineering graduate programs relative representation in 
bachelor’s degrees? How can we better capitalize on high levels of Interest among 
prospective URM majors in the first college year? 

We note three other important points with respect to student diversity. First, senior URM 
women reported the highest mean level of professional and interpersonal confidence and 
the lowest mean self-reported GPA, relative to their senior peers. In the model of seniors’ 
engineering graduate school plans, self-reported GPA was a positive predictor and 
professional and interpersonal confidence was a negative predictor of engineering graduate 
school plans. Are we losing some of these female prospective graduate students, because 
they believe that a lower GPA precludes graduate studies? Do they not see the value of 
social skills in engineering work, or, conversely, do they believe their social skills will have 
greater value in other disciplines? 

A second important point is that senior URM students were more likely to consider multiple 
options that span engineering and non-engineering than were non-URM students (67% vs. 
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56%). This result could suggest that URM students have broader interests, and also that 
engineering as a profession may need to work to retain these individuals among its ranks 
(see Subsection 2.7). 

Finally, our analyses suggest complex interactions between gender and race/ethnicity. 
Issues affecting women are not necessarily the same as those affecting URM students, 
although both demographic groups are underrepresented in engineering. More specifically, 
issues affecting majority (non-URM) women are not necessarily the same as those affecting 
URM women. Given the small number of URM women with advanced engineering degrees, 
these are questions that need to be addressed. They are particularly important as we work 
to diversify engineering faculty. 

Engineering work 

Similar percentages of women and men were planning on engineering and non-engineering 
work and graduate school. However, men were slightly more likely to focus on engineering 
only, while women were more likely to consider both engineering and non-engineering 
options. Further work is needed to understand not only how students conceptualize their 
future careers, but how conceptualizations may vary by gender, particularly given the 
underrepresentation of women in engineering careers. 
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2.10 Enabling Success for Engineering Students: 

A Summary of Research on Student Learning Experiences 

Section 2 of this report described the Academic Pathway Study (APS)— its research design, as 
weil as a sampling of key results. For more information about the APS, the reader is invited 
to the see Section 6 for a description of our research instruments. Appendix A for a fuil list of 
our papers, and the CAFE web site (http://www.engr.washington.edu/caee/) to see research 
briefs on a iarge number of our papers. 

We ciose our discussion on the APS by providing short summaries of six key topics that have 
emerged from the research findings described in the preceding subsections: 

(1) Weicoming Students into Engineering, 

(2) Understanding and Connecting with Today’s Learners, 

(3) Helping Students Become Engineers, 

(4) Deveiopingthe Whoie Learner, 

(5) Positioning Students for Professional Success, and 

(6) Welcoming Students into the Work World. 

After each summary, questions informed by the research are provided to facilitate reflection 
and discussion on how a particular topic piays out on an individual campus and in an 
individual classroom. All of the questions are compiled and presented in their entirety in a 
list entitled “Local Inquiry Questions” that appears in Appendix D. 

For readers in academia, we invite you to consider how these findings might stimulate 
conversation on your campus, and how they might influence how you go about educating 
engineering students. For readers in industry and other areas outside academia, we invite 
you to consider how our findings might be usefui as a guide for helping students get the most 
out of their co-op and internship experiences and for successfuily integrating newiy 
graduated engineering students into the working world. For readers involved with poiicy and 
funding decisions, we invite you to use our findings as a basis for thinking about future 
efforts to buiid and support a vibrant and inciusive engineering educational system. 

In the larger frame, as part of the community invested in today’s and tomorrow’s engineering 
education, we invite your involvement in helping create the best education possibie for 
today’s engineering students. 

2.10.1 Welcoming Students into Engineering 

Perhaps surprisingly, we found evidence that persistence in engineering majors is 
comparable to that in other majors; in other words, students who start in engineering majors 
tend to stick with their majors as much as students in other fieids. Even so, engineering has 
some things to be concerned about. Those who persist— even those who seem to be deeply 
committed to engineering— may have significant and important doubts about staying in their 
engineering majors. Those who leave engineering majors are disproportionately from groups 
underrepresented in engineering, including first-generation coiiege attendees. This results in 
a iess diverse graduating class. In addition, few students migrate into engineering majors 
after starting coiiege, resulting in an over 15% net ioss of students (more than most other 
majors). Low in-migration is partly related to the curricular inflexibility and overloaded nature 
of some program structures. Students who do not begin coiiege as engineering majors need 
to take key prerequisites, which often necessitates extending their undergraduate studies by 
one or more terms. Noteworthy, however, is that some 10% of engineering graduates do 
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migrate into engineering, and that this group has strong representation of underrepresented 
groups (and therefore can contribute to diversifying engineering). 

What we see is that there is not a single pathway into engineering, that opening up those 
pathways less traveled has the potential for broadening participation in engineering, and that 
even those students who seem to be firmly committed to majoring in engineering may have 
doubts about it being the right pathway for them. Students should be encouraged to explore 
and choose pathways through early college experiences that are tied to key motivational 
factors and that let students “try engineering out.” Students can learn about engineering 
through multiple sources— e.g., relationships with faculty, advisors, and peers; coursework; 
co-op/internship experiences; and extracurricular activities. 

Some relevant questions to consider on your campus: 

• Informed Decision Making: Does your college offer courses or programs (such as 
speaker series) that reveal to students the range of Jobs and careers within the 
engineering field? How are students encouraged to integrate a variety of experiences 
into informed decision making on majoring in engineering? Do they have an accurate 
and sufficient understanding of the field of engineering and their place in it? How is 
re-examination of their decisions to stay in engineering supported through advising? 

• Migration in: Are there opportunities in the first years of college at your school (such 
as “introduction to engineering” seminars or courses) that allow students to explore 
engineering? How much migration in is happening at your institution? How might this 
pathway be expanded? Are there institutional barriers that discourage students from 
transferring into engineering? 

• Pathways: What is the range of pathways that your students take through your 
curricula? Where do they find support? What organizations, faculty, student groups, 
and peers help students navigate through the institution? Does your institution 
support varied pathways through the undergraduate experience? 

2.10.2 Understanding and Connecting with Today’s Learners 

Students are motivated to study engineering by a variety of factors, such as psychological/ 
personal reasons, a desire to contribute to the social good, financial security, or even seeing 
engineering as a stepping stone to another profession. Some factors are strong among all 
engineering students— for example, intrinsic psychological and behavioral motivation. Some 
have more influence with one group than another. For example, being motivated by mentors 
is stronger among women, whereas being motivated by the “making” and “doing” aspects of 
engineering (behavioral motivation) is stronger among men. Motivation is related to several 
important outcomes. For first-years, enjoyment of engineering for its own sake (psychological 
motivation) is correlated with intention to complete an engineering major, and, for seniors, it 
predicts intention to enter into engineering work or graduate school. Given these 
relationships, it is important for everyone responsible for engineering education to better 
understand the nature of student motivation and how it might be leveraged to attract a wide 
variety of students to engineering and to provide them with opportunities to explore different 
aspects of engineering. 

Just as motivation to study engineering is not identical for all students, neither is the way 
students construct and experience their college education— /.e., their level of engagement 
with their courses and teachers, and how they combine coursework and extra-curricular 
involvement, as well as co-op, internship, and research opportunities. Some students desire 
significant engagement in everything they do, others are more selective or focused in their 
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involvement, and some are largely uninvolved in out-of-classroom activities. Throughout their 
college careers, women tend to be more involved in extracurricular activities (both 
engineering and non-engineering) and ascribe more importance to these activities than do 
men. These trends also vary with the individual’s levels of psychological motivation and 
confidence in professional and interpersonal skills, and by class standing (first-year vs. 
senior). Of equal importance, even with similar choices, the “lived” experience may vary, in 
terms of a sense of curricular overload or pressure to represent one’s demographic group. 
These findings suggest opportunities for improved advising and curricular program design, 
based on a deeper understanding of what students desire from their college education and 
how they go about constructing and experiencing this education. 

Some relevant questions to consider on your campus: 

• Listening: How do you get feedback from students about the effectiveness of various 
elements of your program? Do faculty listen to students about the effectiveness of 
their teaching? What mechanisms can be put in place to encourage more timely and 
effective use of teaching evaluations by instructors? How can what is learned through 
evaluations be better aligned with program improvement? Do you provide an 
environment where students listen to each other? 

• Student Passion: What motivates students on your campus to choose an engineering 
program? What can they be passionate enough about to keep them in an engineering 
program? Does your program include elements that will ignite and sustain student 
passion? 

• Variability/Commonaiity: How are students in your college of engineering similar to 
one another? How are they different from one another? How well do faculty and 
policy makers on your campus understand similarity and variability in your students’ 
motivation, background, interests, learning challenges, confidence, and future plans? 

• Supporting Diversity: Do individuals from traditionally underrepresented populations 
feel supported and included in the engineering community on your campus? Do 
faculty, students, and administrators recognize and support the important voices 
brought to engineering from individuals of all backgrounds? 

2.10.3 Helping Students Become Engineers 

Students develop an engineering identity and learn about engineering from a variety of 
sources: from co-op and internship experiences, from their coursework and instructors, from 
extracurricular activities, and from personal contacts. We observed that these sources vary 
little by gender or URM status. By senior year, most engineering students in the Longitudinal 
Cohort saw problem solving, communications, teamwork, and engineering analysis as key 
engineering competences and were using more engineering-specific language to express 
technical ideas; this is good news. However, comparing juniors and seniors to first-years and 
sophomores, we saw that the more advanced students did not exhibit greater attentiveness 
to the broad context of engineering design problems (though women considered broad 
context more so than men on some engineering design tasks). 

In addition, seniors did not perceive the broader range of professional and interpersonal 
skills— leadership, public speaking, and business abilities, as well as communications, 
teamwork, and social skills— as being any more important than did their first-year 
counterparts, even having had project-based learning, design experiences, and, possibly, 
co-op or internship experiences. This suggests that the typical engineering curriculum could 
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do more to help students carry what they learn in first- and second-year math and science 
courses into the more engineering-focused classes in their latter years. 

These gaps suggest that some students fail to integrate the knowledge they are gaining 
about engineering from the various sources and across their years of study into a more 
complex, complete understanding of what it means to be an engineer. Furthermore, 
students are not always successful at transferring specific course knowledge and skills to 
real-world problems and settings. For instance, they may not anticipate how the teamwork 
skills they develop in courses using project-based learning are applied when working as an 
intern on a globally distributed design team. Alternatively, they may not recognize that the 
organizational skills needed to manage multiple projects in their co-op assignment are 
similar in nature to the skills they learned in leading a student organization. 

Some relevant questions to consider on your campus: 

• Student Identity as an Engineer: Do the students you teach know what engineers 
really do? Do they identify themselves as engineers? How does your program help 
them do this? Can they articulate what they are bringing to the engineering 
profession? Do faculty and administrators think about a student’s engineering 
identity as an element of student development in the undergraduate years? 

• Connecting Across the Years: Does your college connect the early learning 
experiences in the first two years (math- and science-focused) to the more 
engineering-focused experiences in the later years? How do design experiences in 
upper-division courses build on design experiences in early courses? 

• Learning Engineering: How do you confirm that students have learned and retained 
the basic skills of engineering? Have your students acquired the language of 
engineering? Have they mastered the concepts that are difficult to understand? Can 
they define and solve engineering design problems? Do they have the skills and 
confidence to meet society’s grand challenges? 

• Well-Rounded: How broadly do engineering students on your campus conceptualize 
engineering? How many areas beyond math, science, and analysis would students list 
as important components of engineering? How skilled are your graduates in the many 
aspects of the engineering profession? 

• Designing in Context: Do your graduates have the design skills they need? Do your 
students consider the broad context of engineering problems as they solve them? Do 
they think about the users and other stakeholders of an engineered solution, and all 
aspects of the life cycle? Are they considering global, environmental, societal, 
economic, and cultural context in engineering design? 

2.10.4 Developing the Whole Learner 

Engineering students report experiencing considerable intellectual growth during their 
undergraduate years; they learn to apply key math and science support tools, and learn to 
take on substantial challenges in their design work. In addition, their college studies 
promote gains in confidence in many of the professional and practical skills increasingly 
called for in practice. However, studying engineering may mean students are not able to 
take advantage of other parts of a college education. For example, engineering students 
report lower gains in personal growth and fewer opportunities to study abroad than students 
in other majors. Some engineering students also report a sense of curricular overload. 
Furthermore, graduates report feeling underprepared to take on engineering problems and 
decision-making in real-world engineering practice, where the work often involves 
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multidisciplinary teams, and technical and non-technical factors may be of comparable 
weight. Compared with first-years, seniors are less involved in their engineering courses, are 
less satisfied with their instructors (though they interact with them more frequently) and are 
less satisfied overall with their college experience. In spite of these relative differences, 
seniors reported having significant learning experiences, especially those that were in-depth 
and presented them with a challenge. 

Some relevant questions to consider on your campus: 

• Balance: Are your students satisfied with their undergraduate experiences as 
engineering students? Are they able to balance between their engineering and non- 
engineering extracurricular activities? Is there balance between individual and team 
experiences, well-defined and open-ended problems, and design and analysis 
experiences? Are your students able to find balance between the academic and 
social aspects of their lives? 

• Significant Learning Opportunities: How does your institution provide learning 
opportunities that students consider significant, including experiences that connect 
with what students find meaningful, present students with a challenge, ask students 
to be self-directed learners, give students ownership over their learning, and facilitate 
development of a broad vision of engineering? 

• In-Depth Learning Opportunities: Do your students have opportunities to have 
learning experiences that help them extend their understanding of engineering, e.g., 
internships, co-ops, service learning, research or international experiences, and 
project-based learning? Do you help your students reflect on these experiences and 
integrate them into their understanding of the engineering profession? How might 
these reflections be integrated into program assessment and improvement? 

• Learning Environment: How would you characterize the learning environment on your 
campus? Is there an atmosphere of students in competition with each other? Do 
students feel overloaded by a demanding curriculum? Do all students feel that your 
institution would like them to succeed? Do your students develop confidence in their 
abilities as engineers? Are your students excited when they graduate, or do they 
seem to be Just sticking it out to the end? 

• Asking Questions: What helps your graduates recognize the areas in which they need 
more knowledge or skills? Do they know how to seek out and acquire the knowledge 
and skills they need? Do they feel enabled to continue the learning process after they 
graduate? 

2.10.5 Positioning Students for Professionai Success 

About 30% of the engineering students we studied had post-graduation plans focused 
exclusively on engineering (work and/or graduate school). These students were strongly 
motivated to study engineering for intrinsic psychological reasons and were likely to have had 
co-op and/or internship experiences, in general, these same students were among those 
who were less confident in their professional and interpersonal skills than those considering 
non-engineering professional endeavors post-graduation. 

Most other students conceived of their careers as combining engineering and non- 
engineering components. Some of these students expected different degrees of engineering 
specificity in their work, changing as their careers progressed. Others may still have been 
uncertain, even as graduation approached, as to whether an engineering or non-engineering 
path would be the best fit for them. These patterns might also have been influenced by the 
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focus of the institution that students attended. In any case, faculty, staff, and programmatic 
structures generally do little to acknowledge (much less support and advise) students looking 
at combining engineering and non-engineering endeavors in their career plans. 

Some relevant questions to consider on your campus: 

• Post-Graduation Plans: What resources are available at the department, college, and 
institution levels for guidance in job and career planning? Do your students feel 
enabled to enter a variety of professions? Are they prepared to be effective in those 
professions? What plans do your graduating students have? Are they considering a 
career in engineering, another field, or both? Work in industry or the public sector? 
Graduate school in engineering or another field? 

• Ability to Practice: What challenges do your graduates face when they begin practice 
or graduate school? What helps facilitate their transition? Do they know how to seek 
out the information and advice they need? Are they prepared for their long-term 
careers or Just their first Jobs? Can they effectively communicate their ideas to 
multiple audiences in the many modes they need to? 

• Interdisciplinary Respect: Do your graduates understand the value of skills and 
perspectives from individuals in fields other than engineering? Do they respect both 
other fields and the individuals who practice in these fields? Are they able to work 
with these individuals? 

• Meet Grand Challenges: How prepared are your graduates to take on the wide range 
of roles— in government, industry, and academia— required for engineers to address 
the grand challenges that face the globe and its inhabitants? 

2.10.6 Welcoming Students into the Work World 

Those students who enter the work world after graduating face challenges on multiple fronts. 
They find that the problems that they are solving are more complex and ambiguous than the 
problems that they solved in school. The structures of their new work environments are 
unfamiliar and multi-faceted, and it can be difficult for newly hired engineers to find the 
information they need. Sometimes, they feel that they are not allowed sufficient exposure to 
the “big picture” of where they and their work activities fit into the goals of the work group or 
company. These new hires also find that they are working with larger, more diverse teams 
than they experienced in school— teams that are composed of engineers and non-engineers, 
coworkers, and customers or clients. They must often learn new terminology and new 
communication skills. 

• Practicing Engineering: What challenges do your newly hired engineering graduates 
face when they begin a Job? What can you do to help facilitate their transition? Are 
they supported when they need to seek out information and advice? Are they given 
appropriate orientation, support and mentoring from others in the organization? 

• Working in Diverse Teams: Are your industry-bound graduates prepared to work with a 
wide variety of coworkers and customers or clients in different roles and settings? Do 
they understand the value of skills and perspectives from individuals in fields other 
than engineering? Do they understand that decisions can often incorporate more 
factors than those that pertain only to the engineering aspects? 

• Communicating Effectiveiy: Do your graduates have an appreciation for the needs 
and interests of different audiences when talking about their work or a problem? Are 
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they able to listen to others and effectively incorporate input? Can they communicate 
their ideas to multiple audiences in the many modes they need to? 

2.10.7 Next Steps: Let the Discussions Begin 

We hope the new insights about engineering student pathways gained through APS research, 
coupled with practice-related questions above, will facilitate reflection, stimulate discussion, 
and eventually inform action on your campus. The APS team has already engaged multiple 
communities in productive discussions facilitated in a variety of formats at major confer- 
ences, including the American Society for Engineering Education’s annual conference. 
Frontiers in Education, the Professional and Organizational Development Network in Higher 
Education’s annual conference, and the Women in Engineering ProActive Network’s annual 
conference. Other ways of responding to these findings and questions could be less formal: 
an individual faculty member considering how APS findings might affect how design is taught 
or a conversation between an engineering educator and a faculty development professional 
about effective teaching. Alternatively, it might be an individual student taking greater 
ownership for connecting their work and school experiences. It might also take the form of 
discussion at the department or institution level between faculty, administrators, and staff, 
as part of a broader review of how effectively a particular program is in enabling academic 
and professional success for students on a variety of educational pathways. 

Doing this— understanding the experiences of our students and integrating these insights into 
how we design, deliver, and improve engineering education— will result in education that 
excites more students about engineering and better prepares them to become the technol- 
ogy experts and leaders so needed to address the increasingly complex problems faced by 
the global community. 
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3 Faculty Approaches to Teaching: 

Studies of Engineering Educator Decisions 


In the Studies of Engineering Educator Decisions (SEED) thread of CAEE, we explored 
engineering teaching through a decision-making lens. Emphasizing decision making 
represents a relatively novel approach to studying teaching. Existing approaches for studying 
teaching include research to identify, validate, and promote effective teaching practices, as 
well as research on educator concerns and beliefs about teaching, and the types of 
knowledge involved in teaching. Our focus on educator decisions complements such work. 

A decision represents the point where educator thinking connects 
with educator action, and the decision-making process 
represents a context in which educators can apply research 
findings about students (Turns et al. 2007). Decisions are also a 
site of professional responsibility. Conversations within the 
engineering education community about research-informed 
teaching can be understood as an assertion about one profes- 
sionally responsible way to make decisions. Finally, decisions 
represent a cross-cutting approach to studying how engineering 
students are taught: Decisions that affect students are made not 
only at the classroom level but also in individual interactions with 
students, at the policy level, and in informal contexts (Huang, 

Eliot, et al. 2006). 

In this work, we focused on educators’ decisions by collecting and analyzing educator- 
provided narratives about specific decisions they had made and the processes they used to 
make the decisions. We then analyzed these narratives (hereafter referred to as decision 
narratives) to answer the research questions listed below. 

• RQl: How do engineering educators commit to action in teaching? 

• RQ2: To what extent and in what ways do engineering educators enact effective 
teaching practices? 

• RQ3: What are strengths and limitations in how engineering educators conceptualize 
students? 

Below, we report on the analyses of the decision narrative data we collected from 31 
engineering educators to answer these three questions. In each case, we provide rationale 
for the question, present our approach for addressing the question, share findings from the 
analysis, and discuss implications of the findings. In presenting results, we provide both an 
overall sense of the findings and additional detail (spotlight) on findings of particular interest. 
In discussing implications, we highlight (1) opportunities to recognize educators for what they 
are already doing and to encourage educators to try new things, (2) areas of additional 
research that are suggested by the results, and (3) opportunities for critical reflection 
associated with the results (Brookfield 1995). 

In this section, we also address a fourth question that emerged as we were engaged in the 
work on the first three research questions. We address this research question using data 
that emerged from our efforts to address RQ1-RQ3 and a related synergistic project. 


Decisions represent the point 
where educator thinking 
connects with educator action. 
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• RQ4: How can a focus on decisions and decision narratives be used in teacher 
professional development? 

Our closing comments, which build on all of these analyses, focus on the use of decision 
narratives as a tool for research and professional development. 


3.1 Approach 

This work involved the collection of a data set that was analyzed in multiple ways. In this 
section, we describe how we collected the data, characterize the set of study participants, 
and provide overarching comments concerning the data analyses. Details on the analysis 
procedure for each question are located in the section devoted to the question. 


3.1.1 Data Collection 


Participants were asked to 
identify and discuss two 
teaching decisions. 


We collected data through open-ended interviews with practicing 
engineering educators. The core of the interviews involved inviting 
participants to identify and then discuss two recent, relatively 
challenging teaching decisions. We specifically asked them to 
identify and discuss one planning decision (/.e., a decision made in 
advance of meeting with students) and one interactive decision (/.e., 
a decision made while in the act of teaching). The logic of the 
interview questions was guided by work on the Critical Decision 
Method (CDM) (Klein, Calderwood, and MacGregor 1989). CDM 
represents a method for acquiring information on the ephemeral 
process of decision-making. A main premise of CDM is that the 
decision-making process reported by a participant for a decision 
that is recent, memorable, and potentially difficult will reflect the 

actual process to a reasonable degree. Our emphasis on planning and interactive decisions 
was inspired by the work of Shavelson and Stern (1981). Asking participants to discuss 
decisions represented, for us, a way to get the educators to talk about their teaching in a 
specific and detailed way, rather than in general abstractions. 


Because of a lack of precedent in the use of such interviews with engineering educators, we 
used a pilot study to gauge the reactions of engineering educators to our interview questions 
(Huang, Yellin, and Turns 2006). This pilot work confirmed the feasibility of asking educators 
about teaching decisions, but also demonstrated the need to include warm-up questions to 
help the educators think about their teaching in advance of being asked to select and then 
report on two specific decisions. To address this issue, the interviews also included 
questions about participants’ teaching responsibilities in general. Information on the 
protocol is available in Subsection 6.2.2. 


3.1.2 Participants 

Thirty-one engineering educators at a large public institution in the Northwest participated in 
the research study. These participants volunteered to be in the study by responding to a 
solicitation e-mail that was sent to all engineering faculty in the College of Engineering. 
Participants represented 12 engineering departments at the institution, making up about 
15% of the engineering faculty, collectively. In terms of rank, 12 of the participants were full 
professors with tenure, 7 were associate professors with tenure, 7 were assistant professors 
on a tenure track, and 5 were non-tenure track faculty. Four of the participants had high- 
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level administrative roles within the university in addition to their faculty appointments. Nine 
of the participants were female. All names used in this report are pseudonyms in order to 
protect the identity of the participants. Table 3-A provides detailed demographic information 
on the participants. 

3.1.3 Data Analysis 

Our primary unit of analysis was the participant, and we looked generally at what topics, 
issues, and themes were prevalent across the participants. The specific frameworks used 
for each analysis are presented in subsequent sections along with the findings. 

Clearly, our research was tightly tied to (and limited by) the data we collected. First, because 
the data represented volunteer participants from one educational institution, treating the 
results as a starting point seems appropriate. Second, because the educators were asked to 
select teaching decisions without constraint, our analyses involved necessarily broad 
questions. A final consideration in interpreting the data is the nature of the relationship 
between what was said in an interview and what was “true” (/.e., what they said they believed 
and what they actually believed, what they said they did and what they actually did). 
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Table 3-A: Demographics of educators who participated in SEED. 


Gender Position Department 


femaie 

Assistant 

maie 

Assistant 

femaie 

Assistant 

maie 

Fuii 

maie 

Fuii 

maie 

Fuii 

maie 

Fuii 

femaie 

Assistant 

maie 

Assistant 

maie 

Associate 

maie 

Associate 

maie 

Associate 

maie 

Research Assistant 

maie 

Fuii 

maie 

Fuii 

maie 

Lecturer 

femaie 

Assistant 

maie 

Associate 

maie 

Associate 

femaie 

Fuii 

femaie 

Fuii 

maie 

Fuii 

maie 

Senior Lecturer 

maie 

Fuii 

femaie 

Associate 

maie 

Fuii 

femaie 

Assistant 

maie 

Associate 

maie 

Fuii 

femaie 

Senior Lecturer 

maie 

Lecturer 


Aeronautics & Astronautics 
Aeronautics & Astronautics 
Bioengineering 
Chemicai Engineering 
Chemicai Engineering 
Chemicai Engineering 
Chemicai Engineering 
Civii & Environmentai Engineering 
Civii & Environmentai Engineering 
Civii & Environmentai Engineering 
Civii & Environmentai Engineering 
Civii & Environmentai Engineering 
Civii & Environmentai Engineering 
Computer Science & Engineering 
Computer Science & Engineering 
Computer Science & Engineering 
Construction Engineering 
Eiectricai Engineering 
Eiectricai Engineering 
Eiectricai Engineering 
Eiectricai Engineering 
Eiectricai Engineering 
Eiectricai Engineering 
Industriai Engineering 
Mechanicai Engineering 
Mechanicai Engineering 
Statistics 

Technicai Communication 
Technicai Communication 
Technicai Communication 
Transportation 
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3.2 RQl: Decisions and Commitment to Action 

Our general interest in how engineering educators commit to action in their teaching was 
motivated by an interest in understanding how to heip them make use of information from 
research, such as the information from the Academic Pathways Study (APS). Decisions, as 
points where educators commit to action, represent a context for making use of APS-like 
information or any other information availabie to educators. 

In order to look broadly at the overall process of decision making (Turns, Huang, et a/., 
forthcoming), we analyzed the decision narratives in terms of how the educators did the 
foliowing: 

• Reacting to decision making: The educators’ responses to the notion of decision 
making as a way to talk about teaching 

• Providing rationaie: The educators’ abilities to provide a rationaie for the decisions 
they identified 

• Collecting information: The type of information the educators reported collecting in 
order to make decisions 

• Gauging satisfaction: The level of satisfaction associated with the decisions 

These dimensions are interesting in that it would be easier to support educators in improving 
their teaching practice if they recognized themselves as making decisions (because then 
they could be discussed), could externalize a rationale (because it could then be inspected, 
debated, extended, revised), engaged in more information gathering (because new 
information could be added), and were not completely satisfied with decision outcomes 
(because there would be a reason to iterate). 

3.2.1 Results and Discussion 

Below, we provide overall results concerning the decision making processes reported by the 
educators, followed by a spotlight on how the educators address “time" as a key element of 
their decision-making rationale. 

Overall results 

Reacting to decision making. The transition to the decision portion of the interview followed 
a series of warm-up questions concerning the educator’s teaching responsibilities. Initially, 
most participants reacted to the transition to discussing decisions with a need for negotiation 
about what exactly was meant in order for them to recognize and focus on specific decisions. 
For example, one participant’s initial reaction was, “That’s really hard, because I don’t know 
which type of decisions you’re talking about,” while another stated, “Well, that’s a— that’s a 
big, nebulous question.” However, with clarification from the interviewer that we were 
interested in decisions broadly, all but one participant successfully transitioned to talking 
about a wide range of decisions. Three-quarters of the participants eventually committed to 
specific decisions that became the subject of the interview while the others identified a type 
of decision and talked about the type, but without reference to a specific instance. The one 
participant who was unable to make the transition to talking about decisions stands out 
because of his difficulty in mapping a language of decision-making to his teaching practice. 
Although he was willing to continue with the interview, he commented on multiple occasions 
about the dissonance between the language of decision-making and his own way of thinking 
about his activity. For example, early in the interview, in response to a question about 
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decisions, he stated, “I don’t view it [my teaching] that way.” At a later point, when asked 
whether a particular action could be considered a decision, he clarified, “It’s not a conscious 
decision.” 


The decisions reported by participants were overwhelmingly decisions concerning a group of 
students in the context of a specific teaching assignment, mostly a class. At the same time, 
there were occasional references to decisions concerning interactions with individual 
students, as well as decisions concerning an approach to interacting with students across 
teaching assignments. Perhaps unsurprisingly, no participant reported decisions made 
outside of specific teaching assignments, such as decisions concerning admissions or 
curricular design. However, the absence of such decisions in the dataset is interesting. 

Does this lack of inclusion of decisions outside of specific teaching assignments suggest that 
such decisions are not considered “teaching” decisions, or is it that these types of decisions 
did not seem implied by the way the interview protocol was structured? 


All participants mentioned 
student learning as part of 
their rationale. 


Providing rationaie. All participants were able to provide some 
level of rationale for their decisions and mentioned a variety of 
factors as part of their rationale. Reassuringly, all participants 
alluded to student learning at some level. The types of 
additional factors mentioned by participants inciuded time (31 
of 31), departmental policies (25 of 31), interpersonal 
relationships (21 of 31), student ratings (18 of 31), physical 
space and facility support (17 of 31), funding for themselves 
(15 of 31), keeping themselves engaged (14 of 31), teaching 
assistants (14 of 31), promotion and tenure (14 of 31), 
financial resources for instruction (12 of 31), lack of 
knowledge about a topic (6 of 31), and failing to receive course 
materials from previous educators (4 of 31). There are many 

ways to organize these various factors. For example, while several of these factors could be 
represented as relatively neutral “resource constraints,” a large number of these factors 
were also very specific to the individual making the decision. This broad picture reminds us 
of the complexity of all decisions associated with teaching and how teaching decisions are 
not just about student learning. 


A striking feature of the rationale was the extent to which participants talked about prior, 
related decisions and the influence of those prior experiences on the current decision. Given 
the level of prior experience for most of the participants in our study, this feature of their 
rationale reminds us of the historically situated character of teaching decisions. 


Gathering information. Participants mentioned a variety of 
sources of information as being useful to their decision 
making. For example, they talked about getting information 
from personal experience (31 of 31), colleagues (29 of 31), 
teaching centers (20 of 31), and student suggestions (16 of 
31). While some did refer to making use of research (10 of 
31), these references were vague. These counts represent 
instances where participants were specific about the source of 
their information. An interesting feature of the rationale 
provided by the participants was the number of assertions 
about students that were not linked to a specific information 
source. 


Participants rarely mentioned 
educational research as a 
source of information for their 
teaching decisions. 


Gauging satisfaction. When participants were asked about their level of satisfaction with 
their teaching decisions, five interesting patterns emerged. Participants spoke of not being 
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Dissatisfaction with decision 
outcomes was not aiways 
associated with a desire to 
improve the situation. 


“optimally” satisfied but considering the outcome to be good 
enough (20 of 31), not being satisfied but also not having 
sufficient control to change the situation (19 or 31), not being 
satisfied but simply having to focus on something else (6 of 
31), being satisfied but at personal expense (19 of 31), and 
being satisfied with no reason for additional change (16 of 31). 

Note that these five satisfaction categories are not mutually 
exclusive across participants since participants spoke of at 
least two decisions, and in some cases, spoke of more than 
two decisions. Evidence of emotion was a feature of the 
decision narratives that suggested underlying points of 
satisfaction and dissatisfaction. While the narratives overall 

had a neutral, detached quality, 28 participants spoke of positive emotions at some point, 
and 25 participants spoke of negative emotions at some point. 


A noteworthy feature about their assessments of personal satisfaction was the limited 
Information on which these satisfaction assessments were based. Participants reported 
basing their sense of satisfaction on some form of self-evaluation, such as their own sense of 
student engagement (27 of 31); feedback from students, such as direct feedback through 
student emails or discussions and Indirect feedback through evaluating student performance 
on exams and assignments (25 of 31); and formal student evaluations, such as mid-term 
evaluations or end-of-term student ratings (21 of 31). Participants rarely mentioned peer 
teaching evaluations as a source of information for determining the satisfaction of specific 
teaching decisions (3 of 31). If dissatisfaction with outcomes can motivate change, perhaps 
the addition of information to create dissatisfaction could be a means for Instigating change. 


Spotlight: Time as an element of rationale 

Time stands out as one of the elements of rationale mentioned 
by all participants. What makes time interesting, however, is 
the different ways that the educators in the study talked about 
time (Huang, Yellln, and Turns 2007). As expected, having 
limited time was frequently invoked as a constraint that 
precluded particular types of Innovation, as in this example: 

/ think about the fact that people have different learning 
styles, and I don’t believe in a single learning style. ..It 
would be great if you could teach every idea with every 
learning approach. ..but that’s too time consuming. 


As expected, having iimited time 
was frequentiy taiked about as a 
constraint on innovation. 


However, time was also discussed in more nuanced ways. As 
we discuss below, participants in the study talked about time 

In terms of (1) specific time intervals that affect their teaching decisions, (2) types of time 
such as preparation time, (3) time as something that could be used more effectively through 
creative teaching decisions, and (4) time as a way to take students into account. 


To elaborate, faculty characterized specific time intervals that function as parameters when 
thinking about teaching decisions. Examples of these time intervals included total number of 
hours in a term-long course, the minutes allocated to specific activities during a class 
session, the amount of time needed to grade or assess student work, and how much time 
faculty expected students to spend on specific activities. For example, in discussing how 
many group learning activities to include during the entire course, one participant stated, “I 
mean 50 minutes, one thing, you’re done. And so you can’t do that every week, you just 
can’t.” Another participant, an assistant professor who was beginning his third year of 
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teaching at the university level, referred to the time interval of a term in his discussion of a 
new course that he was developing: “...something that you can teach in 10 weeks that’s 
worthwhile.” 


Preparation time came up for some educators as a type of time of particular significance. 

For example, a full professor who carried a heavy administrative load in addition to research, 
teaching, and family responsibilities described a strategy for minimizing teaching preparation 
time, stating, “I’m sort of proud of myself. In 16 years I’ve only taught 3 different courses, 
which I think is a model of teaching efficiency in terms of prep time.” Another participant, an 
associate professor with eight years of teaching experience, echoed this strategy, describing 
the preparation time needed for a course he had taught several times as having “reached a 
steady state. ..that asymptotic area where my lectures are more or less under control. I 
can. ..show up an hour before lecture, organize my overheads.. .read my notes once, and show 
up, and I’m ready to give a good lecture.” 

New faculty members in our study confirmed how some experienced faculty seemed to 
minimize the time they allocated to teaching preparation. For example, an assistant 
professor with two years of university teaching experience discussed the amount of time he 
spent preparing for his classes, compared with the amount of time that he thought more 
experienced colleagues spent preparing, stating, 

/ probabiy do more preparation for ciasses than a iot of instructors, at ieast the 
instructors i know, who are— some of them are experienced ones. Once you teach a 
ciass iike 27 times, it’s reaiiy easy to wing it. 

However, at least one experienced educator, a full professor with more than 10 years of 
teaching experience, reported spending a significant amount of time on preparation, saying, 
“You need to spend five hours at least to prepare for one good hour of teaching.” 


Some participants talked about 
ways in which teaching decisions 
could be used to create more 
time. 


While time was considered by many to be a significant 
constraint, some participants commented on ways in which 
they essentially created more time for their teaching. For 
example, some faculty members talked about blurring the 
boundaries between their teaching assignments and their 
research interests. Some talked about assigning class 
readings that would match their current research interest. A 
couple of participants talked about including classroom- and 
teaching-related issues into their research agenda (e.g., 
incorporating educational technology, new classroom 
assessment techniques). Others talked about teaching 
classes that only relate to their research interests, enabling 

them to optimize the time spent on both areas. In one specific example, an associate 
professor talked about choosing to teach classes that relate to his research interest and 
background: 


...the courses i tend to choose to teach are courses that are reiated to my 
background, reiated to my interest, either iaboratory or hands-on. The senior-ievei 
course is a new course that i had deveioped—the one i teach in spring— and that was 
based on my interests, started out as just materiais, and graduaiiy evoived toward 
the roie of materiais in construction and constructabiiity, and so the senior course i 
teach is on reinforced concrete construction, and we taik about constructabiiity 
issues— not Just how materiais affect it, but aiso understanding the whoie process. 
But that i guess wouid be a decision, i decided to offer— to deveiop that course and 
offer it because it was kind of a continuation of my evoiving interests. 
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On a different note, another participant, an assistant professor, taiked about using 
alternative teaching approaches such as active learning activities to create time for herself 
during class periods. She then used the extra time to prepare and plan for the rest of the 
class sessions and to think of ways to steer students to the topic of the day: 

Oh, it’s a little bit funny, this active learning, because you— you give out the little 
assignments, and then you all of a sudden have nothing to do, you’re waiting for 
them to think about it, and you want to kind of stay out of their way. And so I kind of 
step back, and sometimes I’ll tidy up the classroom, or I’ll look, you know, and see 
what’s coming up and how I can organize the rest of the class, because sometimes— 
like on this topic, it’s so condensed. I have a content issue with this particular topic, 
and so picking out some of the things that I think are most important. So / guess... 

Tm taking the time of active learning to actually then kind of formulate decisions for 
the remainder of the class. 

Finally, in talking about time, faculty not only mentioned their 
own time, but also issues related to students’ time. This 
involved being respectful of student time (e.g., whether to 
assign projects over major holidays, whether to assign field 
trips on weekends) and how time could be used to ensure 
student success (e.g., how educators in a department could 
work together to distribute exams over time in order to ensure 
students could adequately prepare for each exam, working to 
ensure there is enough time for students to “absorb” material) 

The following example from an associate professor not only 
illustrates an educator attempting to manage student time in 
order to promote success, but also suggests the amount of 
time that can be required for an educator who wishes to 
address such a concern: 

During the junior year, all of the students are taking the same sequence of classes, 
so there are 80 students in the same five classes, and I always tell them, I said, “You 
know, I’m going to work with all the teams that are teaching this quarter so that we 
don’t all give you the midterm on the same day. You know, because that’s like— 
that’s a problem.” And they’re like, “Okay, cool.” And so I send an e-mail to all the 
professors saying, okay, I’m going to give my midterm on the Wednesday of the fifth 
week or the fourth week or whatever. And some of them respond, some don’t. 

And so I had a case where like the professor announces two days before he was 
going to do his midterm on that same day, after I had already asked him, you know, 
to try to work with me on this. And so I communicated with him, and I said, “You 
know, I contacted you with the e-mail, I tried to sort this problem out, and then you 
end up assigning it on the same day.” And he says, “Well, when I was a graduate 
student”— of course, the guy is like 70 years old— ’’When I was a graduate student, we 
had to do all our tests on the same day. There’s no reason why they can’t do it,” 
blah, blah, blah, blah, blah, blah, blah. I’m like this is [...], because you know that if 
they do two midterms on the same day that they’re going to be less prepared for one 
than the other, so, I mean why put them through that? We want them to do as best 
as they can in each class. It doesn’t make any sense. 

It’s Just like— kind of punish them, like a hazing ritual or something like that. And so— 
well, that’s what it really is, you know. And so I went back to the students, and I said, 
“Okay, he changed his date to my date. I talked to him about it, he’s inflexible, so I’m 
changing my date, at last second.” So then I turned mine from Wednesday to Friday. 


Some participants considered 
issues related to students’ time 
when making teaching 
decisions. 
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And then I said, “Don’t tell him I did that, because he’ll change his to Friday, you 
know.” I was just pulling their leg, but, you know, the students appreciated the fact 
that I was willing to work with them, you know. 

3.2.2 Implications 

The educators we interviewed deserve recognition for being abie to articulate a rationale for 
their decisions. It also seems useful to acknowledge that educators may not always be in a 
position to make changes to their teaching, even when they themselves are personally 
dissatisfied. At the same time, while the existence of rationale is a starting point, educators 
should be encouraged to think more critically about their rationale. 

In terms of research, each of the four points of our analysis 
(reaction, rationale, information, and satisfaction) represents a 
point for further research. For example, what is the space of 
actions in teaching for which educators are aware of making 
decisions? What are the limits of this awareness? What 
specifically are desirable qualities of effective rationale? What 
does it mean for an educator to have the capacity to use 
research about students to inform teaching? How do we help 
educators develop the capacity to use research in this way? 

What would happen to educator decision making if there were 
more information on which educators could gauge their 
satisfaction with their decisions? 

In terms of critical reflection, we note that visions of expertise are often associated with 
experts having a great deal of knowledge and expert activity as often involving very little 
conscious reflection. As a result, perhaps expert teachers would be less aware of their 
decisions. From a critical reflection perspective, however, it is important to think about the 
types of assumptions that are embedded in a decision-making process that is tacit and fluid, 
as well as considering which students are benefittingfrom such a decision-making process. 


What does it mean for an 
educator to have the capacity to 
use research about students to 
inform teaching decisions? 


3.3 RQ2: Decisions and Effective Teaching Practices 

Our general interest in the use of effective teaching practices was motivated by the body of 
research that seeks to identify, validate, and share effective teaching practices. Information 
that benchmarks the use of such practices seems helpful to efforts to promote such 
practices. For example, understanding how much and in what ways a particular teaching 
practice (e.g., cooperative learning groups) is being used by educators can be helpful to 
faculty developers wishing to promote the practice, educators considering the practice or 
trying to make sense of their own activities, and researchers seeking to conduct research to 
better understand the practice. Work that focuses on such benchmarking is emerging— e.g., 
Pembridge and Paretti (2010) surveyed capstone instructors about their teaching practices, 
Cox and Courdray (2008) developed a tool for observing teaching practice— but such efforts 
are limited. 

Our initial work in this area started with the observation that the interviewed educators, 
regardless of which decisions they were discussing, made reference to helping students 
connect topics to the “the real world” (Yellin et al. 2007). As we probed this observation, we 
became drawn to the general issue of the ways in which the educators might be positively 
influencing student motivation to learn. This thread led to a second phase of exploratory 
work, in which we sought to map ideas from both theoretical and empirical work on what 
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motivates students to the activities reported by the educators. This inductive analysis (in 
which undergraduate, masters, and Ph.D. students worked alongside research scientists and 
practicing educators) demonstrated the promising nature of looking at our data through a 
motivation lens (Turns, Gygi, and Prince 2009). 

In transitioning to a comprehensive analysis of this issue, we identified the work of 
Wlodkowski and Ginsberg (1995) as offering a broad framework that not only links teacher 
activity to student motivation to learn but also embeds an emphasis on creating 
environments that help all students develop such Intrinsic motivation to learn (Turns, Gygi, 
and Prince, forthcoming). Based on a synthesis of work on motivation, the Wlodkowski and 
Ginsberg framework asserts that educators can help all students feel Intrinsically motivated 
to learn material by engaging in four practices: 

• Establishing inclusion: Creating an inclusive environment through respect toward all 
students and connectedness with students 

• Developing attitude: Helping students develop a positive attitude toward material 
through relevance and choice 

• Enhancing meaning: Helping students to establish meaning of the material through 
engagement and challenge 

• Engendering competence: Engendering student competence through authentic 
assessment procedures that help students see their growing effectiveness in doing 
something the students themselves find to be personally important. 

Analyzing our data relative to this framework consisted of determining, for each practice, the 
number of educators who reported activity that can be mapped to the practice (the 
prevalence) and patterns in how the practice was addressed (e.g., specific approaches, 
reported challenges). Because we did not ask directly in the interviews about either the term 
“motivation” or any of the specific teaching practices listed above, the results may 
underestimate the extent to which these issues are present in educators’ teaching. While 
none of these items were explicitly requested in the interview protocol, the interviews were 
rich with information related to these issues. 

3.3.1 Results and Discussion 

Below, we provide overall results concerning how the educators engaged in teaching 
practices associated with enhancing student motivation. We then spotlight two aspects of 
these results: the ways the educators emphasized relevance (which is associated with the 
“attitude” practice) and the ways in which the educators emphasized connectedness (which 
is associated with the “inclusion” practice). 

Overall results 

The results revealed significant variation in the prevalence of the effective practices 
identified by Wlodkowski and Ginsberg: 

• Establishing inclusion: Connectedness practices were prevalent (mentioned by a 
majority of participants), while respect practices were much less prevalent. 

• Developing attitude: Practices concerning attitude were similarly unbalanced. All 
participants made at least one reference to relevance (an expected finding, given the 
earlier analyses concerning references to the “real world”), while few mentioned 
anything to do with choice or autonomy (with most allusions having to do with student 
choice in projects). 
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• Enhancing meaning: Most participants mentioned practices related to both 
chaiienging and engaging students. References to challenging students were 
frequently to the specific issue of choosing projects or the more generai issue of 
knowing how hard to push students. Practices reiated to engaging students were 
described as serious activities but were also associated with making the class fun. 

• Engendering competence: We found no allusions to authenticity and effectiveness as 
characterized by Wiodkowski and Ginsberg. In fact, there were few allusions to any 
type of effort to help students demonstrate their competence (i.e., few allusions to 
assessment). One might reason that such a finding was due to the fact that we did 
not expiicitly ask participants about assessment. However, if assessment is a critical 
element of teaching, then one would expect such issues to have appeared in a data 
set about teaching. 


Spotlight: Relevance as a pervasive concern 


To iliustrate the findings, we taik first about the practice of 
making instructionai materiai reievant. The prevalence of 
actions reiated to relevance was not a particularly surprising 
finding. Engineering is wideiy considered an applied activity, 
and concerns about the relevance of material would be a 
logical component of the applied nature of engineering. 


All participants made at ieast 
one reference to heiping 
students see the reievance of 
instructionai materiai. 


Reievance was raised by ail of the participants in this study in 
some form. Strategies for making topics relevant included 
(a) thinking about real-world expectations, (b) using projects 
anchored in an actuai exampie, (c) linking the topic being 
discussed to actual examples, (d) leveraging departmental 
core competencies or professionai accreditation requirements, 

(e) basing decisions on what is beiieved to be important in the future and in engineering, and 

(f) considering issues of within-schooi relevance {e.g., this is relevant because it wiil be on 
the test). 


One of the most frequent themes was how facuity emphasized real-world expectations for 
what engineers need to do in the workplace as a way to motivate students to become 
prepared. For example, one participant talked about telling students, “It’s a matter of what’s 
acceptable, professional conduct once you leave this institution, and it’s my Job to prepare 
you for the reai world.” Another participant’s concern for reievance was more behind-the- 
scenes: “I hope we can help students. ..making decision[s] in the face of imperfect or 
incompiete information, which is, you know, iife as a professionai anywhere.” 


The educators in the study also focused on connecting to “reai-world situations” as a means 
of establishing reievance. For example, one participant noted, “Weii, I thought the fact there 
was this obvious case study that people were familiar with wouid also give it a littie more 
immediacy.” Another stated this idea more generally: “I try and tie it in to very— and this is a 
term that is way overused in the universities, but— real-world situations. Our students, a lot of 
engineering students, tend to respond reaiiy weii to that.” 

Another dimension to helping students see relevance involved more direct connection of 
student performance with some aspect of the future activity, such as in this exampie: 

So / said, “[Student name], that was a very nice solution. You’ll be pleased to know 
that’s exactly the way the problem was solved and that led to a patent...’’ So you 
want to build up in the students a confidence that they can solve problems. 
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Spotlight: Connectedness as a surprising concern 

Interestingly, practices related to establishing connectedness 
were (iike relevance) raised by almost all participants in the 
study. Wiodkowski and Ginsberg (1995) explain 
connectedness in the foiiowing way: “Connectedness in a 
iearning group is a sense of beionging for each individuai and a 
felt awareness that one is cared for and one cares for others” 

(p. 63). Practices reiated to connectedness included those 
directly related to caring, as weli as practices related to 
listening and responding to students. 

Multipie participants spoke both directiy and indirectly about caring for students. For 
exampie, one educator asserted directly that he cared about students, although he 
wondered whether students really understood that he cared. Other educators demonstrated 
a caring attitude by expressing concern for students’ emotionai states (e.g., “distinguishing 
the students who have an actual difficulty”) and by expressing concern about fairness. 
Additionai actions linked to caring and connectedness inciuded iearning students’ names, 
keeping in touch with them after the term ends, empathizing with students’ struggies 
(including making comparisons to the educator’s own experience), and being concerned 
about their mental states and stressors. The following passage is an example of an 
articulation of caring: 

There’s no time. And so what am I going to do? But i think i iearned from parenting 
that what to do with that is devote your fuii attention to that student's probiem, to the 
extent that you can, and reaiiy waik them through it, even though it’s not scaiabie. 
Because what they do, they show you a sort of reaiiy dependent, needy side of 
themseives, but if they get the message that you care and wiii be there to back them 
up, then they can sort offiip to this “i don’t need your heip” state. 

Participants aiso spoke of listening and responding: exampies included attending to 
students’ ievel of attention, understanding, and engagement; being responsive to student 
requests for accommodations or extensions; and soliciting feedback through formai and 
informal means. Eliciting feedback and gauging the classroom environment in terms of 
students’ moods and other psychologicai characteristics and then adjusting the activity 
accordingly were frequently discussed activities. 

A majority of participants spoke about listening to students by asking for feedback and 
making the elicitation of feedback a priority. They also spoke of different mechanisms for 
getting feedback. For example, Linda taiked of getting feedback in a relatively expected way: 
by handing out index cards and asking students to provide information on the cards. Other 
efforts to get feedback were more informal, such as one participant’s comments about 
“reading body language, looking at eyebalis. Just you can feei the ciass” to see if the students 
were continuing to follow the lecture. Another participant highlighted the importance of 
prioritizing the act of listening: “You have to be accessible to students, and you have to get 
them to come talk to you, because if they don’t come talk to you, you don’t know what you’re 
doing wrong, you don’t know where you’re faiiing them, and that’s frustrating to me.” 

Whiie iistening can be considered an important part of connectedness, it is arguabiy the act 
of responding to what is heard that contributes to conditions of connectedness. Here again, 
the majority of participants spoke of ways that they responded to what they heard from 
students— such as stopping a lecture and providing an exampie, adjusting class activities 
based on student responses to quiz questions, and changing due dates for a project. One 
participant did express concern about the extent to which an educator can be responsive: 


Practices related to establishing 
connectedness were (like 
relevance) raised by almost all 
participants in the study. 
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So, yeah, I am, too.. .to a fault. I’m much too responsive to my own. ..you know, it’s not 
good for me. I mean l...you know, if you’re sending [discussion forum posts] at 11:30 
at night, and then you’re up on the computer at 6:30 the next morning sending 
[posts], there’s a little bit something wrong, but that’s just how I did it. 

3.3.2 Implications 

it is encouraging that engineering educators are already addressing issues of relevance as 
part of attitude, connectedness as part of inciusion, and chaiienge and engagement as part 
of meaning. At the same time, this analysis suggests that engineering educators should be 
encouraged to think about issues of respect as part of inciusiveness, issues of autonomy as 
part of attitude, and issues of authenticity and effectiveness as part of competence, as these 
issues came up less frequently in the data set. 

Researchers interested in building on this work could pursue 
the issue of autonomy in engineering education by exploring 
what exactly makes autonomy so challenging (e.g., the 
curriculum is too constrained, educators do not believe 
students are capable of autonomy). Similarly, researchers 
could further probe the findings concerning assessment by 
asking, for example, what is the prevaience of the types of 
assessment practices that Wlodkowski and Ginsberg highlight? 

More generally, how should the limited references to 
assessment in our data be interpreted? 

A key issue for critical reflection wouid start with the question 
of whether the educators’ practices are actualiy having the 
intended effect on students (e.g., are students actually seeing relevance, are students 
actually engaged), and move to a more critical focus on the ways in which different groups of 
students are benefitting. 


Researchers interested in 
buiiding on this work couid 
pursue the issue of autonomy by 
expioring what exactiy makes 
autonomy so chaiienging. 


3.4 RQ3: Decisions and Conceptions of Students 

Our general interest in how engineering educators conceptualize students has been 
motivated by the ideas of student-centered or learner-centered teaching as described in the 
teaching literature. Student-centered or learner-centered approaches to teaching are 
considered to be normatively desirable and characteristic of expert teacher activity. 
Connecting to our work, it seems that the ways in which an educator approaches being 
student-centered or learner-centered would be related to how that educator conceptualizes 
students. 

In initial exploratory work, we focused on how a subset of the participants talked about 
students (Turns et al. 2008; Yellin, Turns, and Huang, 2008). Noting that much of the talk 
about students took the form of assertions, we explored the connection between this aspect 
of our data and published work on “personal practical theories” (Yellin, Huang, et al. 2008). 
As we progressed in these exploratory efforts, we were drawn to the prevalence of 
statements in which educators differentiated among students. Since the issue of how 
educators differentiated among students represented a broad topic with potential 
connection to diversity and inclusiveness in teaching, we decided to explore this further 
(Sattler, Turns, and Gygi 2009; Sattier and Turns, forthcoming). 
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In this analysis, we asked the question, How do engineering educators differentiate among 
students in the context oftaiking about teaching decisions? To address this question, we 
developed a conceptual framework consisting of four dimensions by which educators could 
differentiate among students: 

• Knowledge: What students know (e.g., the extent and nature of prior knowledge, level 
of understanding of course content, whether students have misconceptions) 

• Behavior: What students do (e.g., asking questions, coming to office hours) 

• Educational classification: A grouping assumed to be relevant to the educational 
system (e.g., the visual learners, the juniors) 

• Demographic classification: A grouping related to demographic categories (e.g., by 
gender, ethnicity) 

This analysis involved filtering the data for places where the educators had differentiated 
among students (based on linguistic cues) and classifying the instances of differentiation 
according to the four dimensions of the above framework. We then summarized the data by 
determining the prevaience of differentiation based on each dimension (/.e., the number of 
educators who had at least one instance of differentiating based on the dimension), as well 
as patterns in the instances of differentiation associated with each dimension. As with 
motivation, participants were not directly asked about how they differentiated students in the 
process of making their decisions or even how they took students into account in making the 
decisions. Our premise (again, following the Critical Decision Method) was that, if such 
issues were important parts of the process that educators had used to make their decisions, 
they would report on these issues. Indeed, we found that the interviews were rich with 
information related to differentiation. 

3.4.1 Results and Discussion 

All of the educators in the sample differentiated among students at some point in their 
decision narratives. Below, we provide overall results concerning how the educators 
differentiated students in the teaching narratives that they provided. We then spotlight two 
aspects of these results: the pervasive use of academic rank and disruptive behaviors as a 
means to differentiate students. 

Overall results 

The prevalence of differentiating based on each dimension varied. 

• Eighteen out of 31 educators differentiated students based on knowledge. These 
differentiations often involved assertions structured as binaries (e.g., students 
knowing or not knowing something). An interesting feature of these differentiations 
was how generic they often were (e.g., students “getting it” or “not getting it”). 

• Behavior was the most prevalent basis for differentiating among students; 27 out of 
31 educators differentiated students based on behavior. 

• Many of the educators also had at least one instance of differentiating among 
students based on some type of educational classification (22 of 31 educators). 

Much of this type of differentiation was based on class standing or academic rank 
(e.g., freshmen, graduate). Students were also differentiated based on their 
educational pathway and their discipline. 
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• Finally, 14 out of 31 educators differentiated based on demographic classification. 
The specific classifications varied (e.g., “naval officers,” “Chinese,” “Asian,” “ROTC,” 
“Mexican,” “American Blacks,” “foreign students,” and “women”). None of these 
classifications was particularly prevalent; in fact, many of these groups were each 
mentioned by only one educator in our sample. No educator differentiated based on 
socio-economic status. 

Spotlight: Academic rank as a pervasive basis for differentiation 

The prevalence of differentiation based on academic rank (15 participants), while not 
surprising, is arguably interesting. The specific basis of such differentiations did vary: non- 
majors vs. majors, advanced students vs. remedial students, lower classmen vs. upper 
classmen, and undergraduate vs. graduate. However, the most prominent differentiation 
was undergraduate vs. graduate. 

As the educators compared and contrasted the similarities and differences between the 
various populations, they often made assumptions about what students are capable (or not) 
of doing. For example, one educator said, “I think graduate students are a little more flexible 
than undergraduates in terms of what they’re willing to do.” 

Another educator further used academic rank to assess students’ abilities: 

Yeah, their knowledge level and where they are in their degree program. So if I’m 
teaching an undergraduate course for students that are just coming into the 
program, they’re at a different level. They need, in my view, they need some 
stepping stones. 

Similarly, another educator described differences between what students are capable of 
doing based on their academic rank— Juniors and seniors versus freshmen and sophomores: 

/ expect that that there’s going to be a divide. There’s going to be the juniors and 
seniors who already have a lot of experience doing these type— doing this type of 
writing that are going to do fine at this, and then there’s probably going to be a 
couple students who are freshman and sophomores who struggle a little bit more 
with being able to do everything at once. 

Spotlight: Disruptive behavior as a pervasive basis for differentiation 

A somewhat surprising finding was the prevalence of 
differentiation based on disruptive behavior. In all, 18 
educators made differentiations among students that involved 
disruptive student behavior. Further, when the educators 
started discussing these behaviors, they tended to focus on 
the event, returning to the issue multiple times throughout the 
interview. For example, educators described disruptive 
behaviors of “talking,” “giggling,” “being argumentative,” 

“being resistant,” as becoming or being a “distraction” and 
“disruption.” Most of the time, these assertions were quite 
general, without elaboration or further development (e.g., 

“argumentative”). Others were quite specific and descriptive in 
nature: “The things that distract from the flow are arguments about 
point on this,’ or ‘Why did you knock off this’...” 


In all, 18 educators made 
differentiations among 
students that involved 
disruptive student behavior. 


...‘I needed a half a more 
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3.4.2 Implications 

The resuits of this work suggest that faculty developers and others that work with 
engineering educators shouid recognize that engineering educators are sensitive to some 
ways in which students differ, since being aware of and responding to student differences 
represents a form of student-centeredness. At the same time, the results suggest that 
educators should be encouraged to think more about the ways in which differentiating 
among students based on student knowledge could be made a more central element of 
teaching practice. 


To what extent do engineering 
educators consider a range of 
explanations for student 
behaviors? 


This research suggests a variety of areas for future research, 
ali of which are related to criticai refiection issues of fairness 
and equity. For example, researchers could further explore the 
issue of what educators believe to be important characteristics 
of students at specific academic leveis. Additional questions 
in this area could include the following: How aware are 
educators of their beliefs? How do educators take into 
account the idea that those beliefs do not apply to all 
students? To what extent do such beliefs create seif-fuifiliing 
prophecies, because educators make instructional decisions 
based on their beliefs? Also suggested by our data, 

researchers could explore the extent to which educators consider a range of explanations for 
student behavior (e.g., why are students behaving in ways that are being labeied disruptive). 
This type of analysis could probe into what educators are assuming about student behavior 
when they are not explicitly seeking explanations, and how such assumptions might 
disproportionateiy impact some students. 


Researchers could also explore a possible explanation for part of the pattern in our analysis: 
the possibility that educators differentiated among students based on what is most visible to 
them (e.g., behavior, academic rank). If visibility explains the pattern of differentiation we 
saw, then educators could be supported in more sophisticated differentiation efforts by data 
that makes visible the basis for other differentiations. Finaiiy, another line of research could 
seek an explanation for the lower prevalence of differentiating based on knowledge. While 
one could sustain that the limited prevalence of differentiation based on knowledge was the 
result of our protocol (/.e., we did not ask about differentiating based on knowledge, and 
perhaps the educators just identified decisions for which knowledge was not relevant), such 
a position seems, at minimum, worth closer examination, since student knowledge is 
arguably fundamental in ali teaching. 


3.5 RQ4: Decisions and Teacher Professional Development 

in starting this work, we did not set out to document how the process of collecting and 
analyzing decision narratives couid itseif be a form of professional development. 
Nevertheless, information in this area emerged as we conducted the work. In this 
subsection, we identify three stakeholder groups who may have benefited from this research: 
the researchers who analyzed the data, the researchers who collected the data, and the 
educators who were interviewed. For each stakeholder group, we identified a data source 
that may contain insight, and we explored the data for links between the data and general 
issue of professional development concerning teaching. 
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3.5.1 Benefits to Those Individuals Who Analyze Data 

We involved a number of people in our data analysis effort. In particular, at two points in the 
process, we organized a for-credit research group on the University of Washington campus in 
order to enlist the aid of various students (Larson et al. 2009; Gygi, forthcoming). As it 
turned out, this not only extended our capacity for analyzing the data, but also gave us a 
chance to think about how analyzing such data could help participants gain insights into 
teaching. 

As a final assignment in the second for-credit research group, participants reflected on what 
they had learned as a result of participation in the group. Further, because the group had 
been advertised as a general opportunity to learn to analyze qualitative data, it was not 
expected that participants would talk about what they learned about teaching. However, six 
of the ten participants alluded to insights concerning teaching. 

One of the three Ph.D. students, a student with a significant 
amount of teaching experience, included insights concerning 
teaching in her reflection. In particular, she reported that the 
process gave her a “new perspective on engineering 
educators.” She also reported that the opportunity gave her a 
chance to reflect: “I found myself often reflecting on my own 
teaching decisions as I read the transcripts and questioning 
how I motivated my students or considered their differences." 

Two of the three masters students in the group included 
insights concerning teaching in their reflections. One reported 
that she now saw “educators as designers” and had become 
“more reflective about [her] own teaching practices.” The 
second student (a student with no teaching experience) reported on how the activity had 
caused him to recognize and reflect on a variety of his own assumptions about teaching and 
his own educational experiences. In his words. 

To be honest, I just thought that there were good instructors and not so good 
instructors... i have aiways considered motivation, especiaiiy academic motivation, 
as an intrinsic quaiity that was either an advantage or disadvantage for individuai 
students. Aiso, i never assumed that instructors and professors cared about student 
motivation... i was surprised in the way that the instructors in the transcripts 
discussed student motivation, even though the word was rareiy mentioned directiy. 
Student motivation was a deeper concept than i initiaiiy considered, and it made me 
think of some of the assignments, interactions, and decisions made by professors 
and instructors during my undergraduate days. 

Three of the four undergraduate participants also made comments concerning teaching. In 
contrast to the experiences of the graduate students, which involved reflecting on their own 
experiences, the comments of the undergraduates emphasized simply being exposed to, and 
reassured by, the teachers’ perspectives. For example, one stated that “it is interesting to 
know that some really care about the students.” A second noted that 

...as a student, i aiways had the opportunity to rate the teachers and just sit back 
and not know whether any of my suggestions were being impiemented or not. 

However, by being a member of this research, this time, i got to sit back and read 
what the teachers said and whether any of the suggestions got impiemented at aii. 

Finally, a third undergraduate noted that “the transcripts were a fascinating way to examine 
what teachers were doing, if anything, to motivate student learning.” 


"/ found myself often reflecting 
on my own teaching decisions as 
I read the transcripts and 
questioning how I motivated my 
students or considered their 
differences. ” 
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In follow-up interviews, Gygi (forthcoming) found that all of the doctoral students found the 
experience to provide an opportunity to reflect on teaching and also an opportunity to identify 
new practices to explore. In fact, two of the doctoral students participated in the group 
because they wanted to learn more about teaching. One commented that the interviews 
were an exceilent source of teaching insight, and the opportunity to analyze the transcripts in 
a collaborative environment was a valuabie professional development opportunity. 

3.5.2 Benefits to Those Individuals Who Collect as Well as Analyze Data 

Given the emerging insights into how the analysis of decision narratives was helping 
participants gain insights into teaching, we decided to explore the feasibility of having 
graduate students collect such narratives. Thus, as part of a graduate course calied 
“Advanced Studies in Design Thinking and Knowing” (taught by Dr. Jennifer Turns and Dr. 
Mark Zachry), we asked ten students to identify design educators who they admire and 
interview the educators about their approach to teaching design, including key decisions that 
they had made. As a class, we constructed an interview protocol and then helped class 
participants make decisions about who to interview. Then, each class member worked to 
arrange and conduct an interview, prepare and present a short report on what had been 
learned, and then participate in a discussion concerning the results. 

All students successfully navigated the logistics of the activity: identifying educators, setting 
up interviews, and getting their respondents to talk about their teaching. In subsequent 
reflections on the activity, students reported not only on their enthusiasm for the activity 
(e.g., “I thought the discussion following our interviews was really interesting, and it’s too bad 
we can’t take those questions we all wrote up and discuss further.”), but also on the specific 
kinds of insights that they gained from their individual interviews and collective discussion. It 
appears that the interviews raised fundamental questions for the students such as the 
following: What counts as design? (“One of the ideas that stuck in my head was that of what 
constitutes design,” and “how hard of a time I had fitting [this educator]’s ideas of design 
into our reading— before the interview I expected this process to be simple.”) To what extent 
can design be taught? What are best practices for design education? Are tools and general 
techniques for design education separable or inseparable? 


3.5.3 Benefits to Interviewee 


If both conducting and analyzing the interviews can play a role in professional development 
concerning teaching, then it is valuable to consider the extent to which simply being 
interviewed can serve a professional development function. 


Since we did not have this question in mind when we conducted our interviews, data to 
support this proposition are limited to statements volunteered by participants. A small 
amount of such data does exist. For example, one participant stated at the end of the 
interview that “it helps me think about my teaching to interact with someone like you.” 
Another participant was not as direct but did note, “I think it’s a 
very interesting interview, and the questions you raised are 
very key and very interesting.” Another participant, who also 
helped our recruiting by distributing our solicitation e-mail, 
added the following when she distributed the solicitation: “I 
did one of these interviews, and the opportunity to talk with 
[the interviewer] was invaluable.” More generally, it was 
interesting to see how many people volunteered to be in the 
study, and how generous they were with their time. 


“I did one of these interviews, 
and the opportunity to taik with 
[the interviewer] was invaiuabie: 
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While such evidence is clearly anecdotal, there is substantial theoretical reason to be 
interested in the benefit of such interviews to the participants themselves {e.g., Brookfield 
1995). As a result, future researchers who engage in this type of research are encouraged to 
consider exploring this in greater detail. 


3.6 Summary 

Decision narratives collected with the Critical Decision Method are powerful tools for 
advancing teaching in engineering education. In this body of work, we used decision 
narratives to investigate multiple questions: 

• In analyzing the decision narratives to better understand educator decision-making 
(see Subsection 3.2), we found that most of the educators we interviewed reacted 
positively to the emphasis on decisions and decision making, and that all were able to 
provide a rationale for their decisions (with both time and allusions to prior decisions 
as common features of their rationale). We also learned that the participants 
collectively mentioned a variety of sources of information as being useful in decision 
making (although research was infrequently mentioned as a source), and we identified 
five patterns in terms of satisfaction with their teaching decisions. 

• Looking beyond their decision processes and towards what additional information the 
decision narratives could reveal, we analyzed the narratives to explore educators’ use 
of teaching practices that are considered effective at helping students develop 
intrinsic motivation to learn (see Subsection 3.3). In this analysis, we found that 
engineering educators reported using a variety of teaching practices that are known to 
increase student motivation to learn, such as helping students see the relevance of 
material, helping students feel connected to the learning group, and helping students 
experience productive levels of engagement and challenge. We found less frequent 
mentions of providing students with opportunities for autonomy, enabling all students 
to feel respected, and providing students with opportunities to demonstrate their 
growing competence. 

• Driven by the broad issue of how engineering educators conceptualize engineering 
students, we analyzed the decision narratives to learn more about how engineering 
educators differentiate among students (see Subsection 3.4). In this analysis, we 
found that all of the educators differentiated among students at some point, that 
student behaviors were the most prevalent basis for differentiating among students, 
and that differentiation based on other dimensions (what students know, their 
educational classifications, their demographic classifications) was also prevalent but 
less so. 

• In addition to these analyses, we also investigated the benefits of engaging in 
research on teaching decisions (see Subsection 3.5). Here, we reported evidence to 
suggest that engaging in research on teaching decisions has professional 
development benefits for the individuals who analyzed the decision narratives, the 
individuals who collected the narratives, and even the educators who were asked to 
provide the narratives. 

In our analyses, we focused on broad questions because we collected data on a broad set of 
teaching decisions. Innumerable variations of this work could focus on collecting decision 
narratives relative to a wide variety of specific constraints (e.g., decisions about assessment, 
decisions about student projects, decisions about working with first-years). 
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We propose that the outcomes of this research may be useful for faculty development 
personnel in helping them to better understand their faculty clients. Additionally, decision 
narratives themselves might be used by faculty developers to initiate fruitful discussions with 
faculty on problematic teaching issues. 

An exciting feature of the use of decision narratives in research is the idea that the process 
of doing the research is potentially significant for all parties involved (Subsection 3.5). Our 
tentative results about these benefits should be further validated. 

In moving forward, there are opportunities to bring these ideas together— for projects 
featuring not only collection and analysis of decision narratives in specific domains but also 
active efforts to leverage (and document) the collection and analysis activities as 
professional development efforts. 
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4 A Focus on Teaching: 

Engineering Teaching Portfolio Program 


The Engineering Teaching Portfoiio Program (ETPP) was designed to assist future engineering 
facuity (primariiy graduate students and post-doctorai researchers) in deveioping a teaching 
portfolio. The program was aiso designed to encourage participants to reflect on teaching, 
which could in turn increase their knowledge about teaching, alter their attitudes, and iead to 
changes in their current or future teaching practices. 


ETPP was designed to encourage 
future engineering facuity to 
refiect on teaching. 


This work was motivated by nationai interests in promoting 
effective teaching in engineering and in better preparing future 
facuity for teaching responsibilities. In addition, our emphasis 
on preparing graduate students for teaching responsibilities is 
similar to the emphasis of two other components of CAEE: 
preparation of future engineers (Academic Pathways Study) 
and the preparation of future researchers (Institute for 
Scholarship on Engineering Education). In developing ETPP, 
we sought to complement other means by which graduate 
students become prepared for teaching careers (e.g., actual 

teaching assignments, campus-levei courses on teaching, workshops associated with 
professional meetings, how-to books). We also sought to develop an experience that wouid 
be valuable to all participants, regardless of their teaching experience. 


This work consisted of an iterative, research-based design process; program offerings at two 
universities; and a formal program evaluation expioring impact on participants. The primary 
work with the Engineering Teaching Portfolio Program spanned the period from 2003 to 
2006. Interest in this program (as measured by downloads of the materials from the CAEE 
web site) continues to this day. 
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4.1 The Program and Its Development 

The Engineering Teaching Portfolio Program provides participants with the opportunity to 
examine, reflect, and revise their beliefs and goals as teachers through a series of eight 
weekly meetings consisting of portfolio development exercises and peer review of drafts of 
portfolio components. The exercises include identifying portfolio design specifications, 
developing and revising a teaching statement, identifying and annotating teaching artifacts, 
developing a diversity statement, compiling a complete draft portfolio, and outlining a 
professional development plan. The ETPP is designed to be peer-led and peer-focused. 
Participants rotate the leadership role for each session and facilitate the sessions without 
supervision by faculty or professional staff. 

Key features of this program include the following: (a) a focus on graduate students, (b) a 
series of activities that collectively help students develop a teaching portfolio, and (c) a peer- 
led structure. The program contains these elements for three primary reasons. First, we 
have an interest in helping improve the flow of graduate students Into faculty positions In 
engineering. The program content and products are designed to meet the needs of 
graduates entering the academy. Second, our goal Is to develop a scalable model for 
working the pipeline issue nationally. The peer-facilitated structure Is cost-effective, because 
it does not require significant institutional commitment of fiscal or human resources. Third, 
engineering graduate students are more likely to participate in a program that has been 
customized to engineering. Our program is product-oriented and designed to help prepare 
participants for the academic job search. 

Two additional distinctive features of our program are the emphasis on peer review as a key 
program activity and the requirement that participants create a diversity statement. 
Concerning peer review, the program includes time for participants to engage in peer review 
of their teaching philosophy, their annotated artifacts, and their diversity statements. This 
emphasis on peer review is not simply a means to help participants improve their documents 
but also a context for participants to share experiences, a means to trigger meaningful 
discussion about experiences, and an opportunity for participants to question fundamental 
Ideas related to teaching. The diversity statement is also a defining feature of the program. 
Participants are asked to discuss diversity in teaching, prepare a statement concerning how 
they address diversity in their teaching, and then engage in peer review of these statements. 
A review of teaching portfolio initiatives at other institutions around the nation suggested the 
distinctiveness of these program elements (Yellln etal. 2006). 

4.1.1 Development Process 

We used an iterative, research-informed process to develop the program’s curriculum and 
supplemental materials. Data collected during program offerings provided the basis for case 
studies of the peer review processes and the activities related to the diversity statement. 

The data also allowed us to gain insight into participants’ conceptions of central topics (/.e., 
teaching and diversity). We used information from these analyses to refine the weekly 
guides and develop the supplemental materials. We also published the results of these 
analyses, so they would be available to others engaged In similar work. These studies are 
briefly summarized below: 

• In (Turns, Yellln, etal. 2006), we presented a detailed analysis of the conversation of 
one ETPP group during the diversity statement components of the program (/.e., the 
introduction of the diversity statement and the subsequent peer review of 
participants’ draft diversity statements). In particular, we documented the content of 
their conversation concerning diversity, teaching, and the intersection of these issues. 
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Topics included diverse populations, diversity and activities in industry, diversity and 
activities in academia, attitudes about diversity, and strategies for and challenges of 
addressing diversity in teaching. We also presented evidence on the interweaving of 
these topics overtime (e.g., interweaving of diversity in generai, teaching in general, 
and diversity as related to teaching) and the inclusive, balanced nature of participant 
contributions overtime (/.e., all participated, no one dominated). 

• In (Sattler, Yellin, et al. 2007), we discussed trends in how program participants talked 
(during post-program interviews) about diversity and their reactions to discussing 
diversity during ETPP. Through the analysis, we noted how participants’ experiences 
about diversity were frequently related to critical incidents in their prior experience 
and how issues of “political correctness” manifested in the ETPP conversations about 
diversity. 

• In (Turns, Yellin, et al. 2006), we presented a detailed 
analysis of the content of the peer-review conversations of 
two groups during the second and seventh weeks of the 
program. We found that participants discussed choices 
and assumptions about audience, negotiated 
understandings of the portfoiio genre, shared general 
writing strategies, exchanged editorial feedback, shared 
specific ideas related to teaching, provided each other with 
positive reinforcement, grappled with issues of “stage 
fright” about their writing, and engaged in overall task 
negotiation. The content of these conversations provided 
evidence for claims that participants were gaining new or 
refined ideas related to teaching, as well as improved 
teaching portfoiio documents, confidence in their 
documents, and general ideas about how to leverage 
feedback in writing activities. 

• In (Huang, Yellin, and Turns 2005), we documented the broad range of ideas about 
teaching that were introduced through conversations during program sessions. These 
ideas— ideas about what constitutes teaching, what counts as “good” teaching, the 
role of decision making in teaching, and a variety of issues related to pedagogy and 
assessment— provided a basis for thinking about how ETPP could increase or refine a 
participant’s teaching knowledge. 

4.1.2 Program Materials 

The ETPP program materials (overviews, weekly guides, and supplemental materials) 
represent a significant product of the development of the program. The overviews and 
weekly guides describe the key activities, while the supplemental materials contain 
information to address issues that came up during initial offerings or were raised during 
discussions with our advisory board. These materials are available online 
(http://www.engr.washington.edu/caee/etpp-sessions.html). 


In addition to discussing 
portfolios and teaching, 
participants provided each other 
with positive reinforcement, 
grappled with “stage fright" 
about their writing, and engaged 
in overall task negotiation 
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4.2 Offerings at the University of Washington and Beyond 

Over the period from 2003 to 2006, approximately 100 people voluntarily participated in 
offerings of the Engineering Teaching Portfolio Program (Table 4.2-A). These counts focus on 
active participants. The number of people expressing interest was much higher, as was the 
number of people attending initial sessions. The most common reason for not enrolling or 
actively participating was scheduling. These offerings were held at the University of 
Washington and the University of Florida. Additional spinoff activities expanded the number 
of people influenced by the ETPP effort. 

University of Washington, 2003 

The first offerings of ETPP took place in the summer of 2003. Graduate students from all ten 
departments in the College of Engineering at the University of Washington (UW) were invited 
to participate in the program. Participants were fifteen graduate students organized into two 
groups. This initial offering involved decentralized, peer-led facilitation, where participants 
each signed up to facilitate at least one of the eight sessions. To support formative 
evaluation, data collected during these sessions included field notes for each session for 
both groups (including real time information on who said what), a group interview with each 
of the two groups, interviews with individual participants, and surveys of individual 
participants. 


Table 4.2-A: Iterations of ETPP, along with number of participants. These numbers 
represent those participants who completed the program. Initial enrollment numbers 
were higher (total of approximately 140). 


Iteration 

Number of 
active 

participants 

University of Washington, Spring 2003 

18 

University of Florida, Fall 2005 or Spring 2005 

9 

University of Washington, Summer/Fall 2005 

39 

University of Washington, Summer 2006 

28 

Total 

94 


University of Fiorida, 2004-2005 

ETPP was used at the University of Florida (UF) in the fall of 2004 and again in the spring of 
2005. This adoption by another school was due primarily to the efforts of a postdoctoral 
researcher in aerospace engineering at UF who learned about the program through an ASEE 
paper reporting on the UW pilot program. Although ETPP was still being polished, the UF 
researcher began a recruiting effort that garnered an initial group of 35 interested students 
(nine completed the full eight-week program). Her wiilingness to implement a program that 
was essentially still in development enabled the ETPP team to fine-tune the program more 
rapidly. Details of these offerings were published at the 2005 ASEE conference (Rosea and 
Hickey 2005). 
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University of Washington, 2005 

The 2005 offering of ETPP at the University of Washington featured a larger number of 
participants than the previous offerings, a curriculum that had been revised based on the 
formative evaluation efforts, and a collection of supplemental materials made available to 
provide participants with addition support. Twenty-five participants, divided into four groups, 
participated in the summer of 2005, and fourteen additional Individuals participated in the 
fall of 2005. As in previous offerings, these offerings involved decentralized peer-led 
facilitation— participants each signed up to facilitate at least one of the eight sessions. To 
support additional formative evaluation of the program, we collected data during the summer 
2005 offerings (field notes for each session, interviews with participants). No additional data 
was collected during the fall 2005 sessions. 

University of Washington, 2006 

The summer 2006 offering of ETPP featured small changes to the curriculum and 
supplemental materials, as well as a revised approach to facilitation. The 28 participants 
were organized into two groups. In response to challenges associated with the decentralized 
peer-facilitation approach used in previous offerings, these offerings featured a common 
facilitator. In keeping with the peer-led approach, the facilitator was another graduate 
student who had been hired on the basis that she would help the group proceed through the 
activities. There was, however, no expectation that she would or could provide expertise 
concerning teaching portfolios. Data collection for these offerings was limited to reflections 
made by the facilitator. 

ETPP variation. University of Washington, 2005 

In 2005, a variation of ETPP was used with undergraduate students who were being trained 
to be mentors for the Minority Science and Engineering Program at the University of 
Washington. This work, which involved four undergraduate students, was documented in an 
ASEE conference paper (Yellin et al. 2005). One finding from this work was that, like the 
graduate students and post-docs, undergraduates have strong ideas about teaching, even 
though they may not be able to express them using formal terminology. 
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4.3 Impact on Participants 

An evaluation of ETPP was conducted by staff at the University of Washington’s Office of 
Educational Assessment, with an internal report completed in 2008 (Maring). This 
subsection highlights the findings from that evaluation, specifically findings based on 
interviews with participants of the 2005 University of Washington offerings (n = 28) and a 
follow-up survey sent in 2007 to all ETPP participants. The evaluator invited participants of 
ETPP to take part in an online survey about longer-term impact of the ETPP experience. A 
total of 89 participants were successfully contacted, and 43 responded to the survey (48% 
response rate). 


Table 4.3-A: Participants’ ratings about ETPP overall on the follow-up survey 


Survey item statement 

% of participants who 
agreed or strongly agreed 

As a whole, 1 enjoyed my ETPP experience 

86% 

In hindsight, 1 think my decision to take part in ETPP was 
a good one. 

88% 

At this point, 1 consider ETPP a good use of my time. 

86% 

1 would recommend ETPP to other graduate students. 

88% 


In general, participants were very positive about their overall ETPP experience. In the 
immediate post-interviews, participants praised the program, noting various ways in which it 
was valuable. On the follow-up survey, participants remained positive. 

Table 4.3-A shows the percentage of participants who agreed or strongly agreed with positive 
evaluative statements about ETPP as a whole. 

In terms of specific outcomes, the evaluation provided evidence that participants created a 
tangible product (/.e., the portfolio) that they valued and, in some cases, used in job 
applications. They reported that building the teaching portfolio and peer discussions tied to 
the construction of the portfolio served as mechanisms for them to think more deeply about 
their teaching, to articulate the decisions they make while teaching, and to formalize these 
thoughts. Concerning the diversity elements, they were mixed on the benefits and impacts of 
talking about diversity. Ultimately, respondents indicated (in the interviews) that the ETPP 
had resulted in changes in their knowledge along a range of topics: about the design and 
use of portfolios, about conceptions of teaching, and about approaches to diversity. Reports 
of direct impacts on teaching were mixed, but this makes some sense, since most 
participants were graduate students who had little formal teaching experience. 

Nevertheless, many felt that the program had had or would have a positive impact on their 
teaching. Additional highlights of the evaluation are included below: 

• The majority of participants indicated on the follow-up survey that they had used a 
part of their portfolio (73%) or looked at it (77%) at least once since they took part in 
the program. These findings indicate the long-term utility of the final ETPP products. 
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• In the immediate post-ETPP interviews, almost half of the participants (41%) noted 
that increased reflection about teaching was one of the most important effects of 
being in the program. More specificaliy, participants appreciated the opportunity to 
formalize thoughts about teaching that they had previously had in an informal or 
disorganized way (e.g., “while lying in bed”). In the follow-up survey, participants also 
made comments about ETPP making them “think more” in generai about teaching, 
but with iess frequency than participants in the immediate interviews. 

• Both immediately after ETPP and at the follow-up survey, participants most frequentiy 
mentioned the discussions with their peers in the ETPP sessions as the most 
significant and/or valuable aspect of the program. One specific way in which the 
discussions were valuable was in gaining new perspectives on teaching. Participants 
commented that they appreciated hearing and reading about others’ decisions, 
strategies, and teaching methods in general. Seeing documented examples (via the 
teaching portfolios) of what others do in their teaching was noted as particularly 
valuable by some participants. 

• Another aspect of discussion that was perceived as 
particuiariy vaiuabie was receiving feedback about the 
components of the teaching portfoiio. Both immediately 
after the program and at the follow-up, participants 
commented that they appreciated hearing how their 
teaching portfoiio documents could be improved. In 
general, participants commented that these discussions 
were particularly fruitful, because they were with peers 
and not with more seasoned teachers or faculty members. 

This may be because the discussions gave participants the 
opportunity to practice expressing their ideas in a iow- 
stakes environment. 

• The majority of respondents (74%) indicated that the statement, “Whenever it is 
feasible, I try to document the things I do as an instructor,” was more true because of 
ETPP (ratings of 4 or 5). In the interviews conducted immediately after ETPP, 
increased documentation of teaching decisions and activities was the most frequently 
mentioned change to actual teaching practices. 


Participants commented that 
discussions about improving 
portfoiios were particuiariy 
fruitfui for being with peers, vs. 
more seasoned teachers. 
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4.4 Summary 

The Engineering Teaching Portfolio Program efforts resulted In a comprehensive set of 
curricular materials, research that not only Informed the development of the curricular 
materials but is available to others interested in supporting graduate students, around 100 
program “graduates,” and several small-scale spinoff efforts. ETPP is notable in the way it 
embeds opportunities to learn about teaching In the production of something inherently 
desirable to future faculty (the portfolio), features conversations about diversity, and involves 
a way of talking about teaching that supports participation by people with a wide range of 
prior experiences. 

Potential future work with ETPP includes additional offerings at the University of Washington 
and beyond, integration of APS results Into the curriculum and supplemental materials, and 
more extensive analysis of the data to fully understand how participants benefit from 
program participation. Concerning the issue of benefits, a promising hypothesis is that the 
educational power of portfolio construction comes from consideration of the significant 
questions that can be associated with portfolio construction (e.g., who am I talking to, what 
exactly do I want to say about my teaching, who judges teaching, how do I provide evidence 
of my strengths as a teacher, what counts as “good” teaching). Stakeholders Interested In 
this issue are encouraged to look Into work on critically reflective teaching (e.g., Brookfield 
1996) and to pursue critical Incident approaches to data collection (e.g., asking participants 
what surprised them during a session, what they were skeptical about during a session, 

“aha” moments, etc.) as a means to tap further Into such issues. 
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5 Building the National Community of 
Engineering Education Researchers: 
Institute for Scholarship on Engineering 
Education 


The primary goal of the Institute for Scholarship on Engineering Education (ISEE) component 
of CAEE was to cultivate a diverse community of engineering education researchers who can 
think and work across disciplines with the ultimate aim of improving the engineering student 
experience. A secondary goal was to formulate principles and models for advancing this 
community of scholars. Both goals address a mission of building capacity (people and 
models) In engineering education. 

As part of accomplishing these goals, three cycles of the Institute for Scholarship on 
Engineering Education were held. The three cycles Involved 49 engineering education 
researchers (Institute “Scholars”) representing 20 institutions (Table 5-A). Recruitment of 
ISEE Scholars sought to represent the range of diversity within the national academic 
community: 20 (40%) were women, and 17 (36%) were underrepresented minorities. All 
faculty ranks were represented: 6 full professors, 12 associate professors, and 9 assistant 
professors. The remaining participants held other academic roles: 13 graduate students 
and 9 participants in other roles. Including staff and administration. 


5.1 Institute Organization and Themes 

Each ISEE cycle consisted of five main phases: (1) designing and/or adapting the Institute 
model, (2) recruiting Scholars for the current year’s Institute, (3) a week-long Summer 
Summit kick-off event at the host school, (4) activities during the academic year to support 
Scholars In conducting their studies, and (5) a culminating Leadership Summit event. 

The week-long Summer Summit was designed to engage the Scholars In the process of 
engineering education research, introducing many to new techniques and ideas in 
educational research. Activities and discussions during the Summit helped Scholars refine 
their research questions, decide on appropriate methodology, and, very Importantly, to form 
a community that could be sustained beyond the week-long meeting. After the Summit, 
Scholars typically returned to their respective campuses to conduct their research, with 
frequent electronic communication and interaction with fellow Scholars and the ISEE team. 
Approximately one year after the Summer Summit, Scholars participated in a Leadership 
Summit at an engineering education conference. (Adams et al. 2006) 
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Table 5-A: Primary institutional affiliation of the ISEE Scholars (with number of 
participants in parentheses) 


2004-2005 2005-2006 2006-2007 


California Polytechnic State 
University, San Luis Obispo 

Howard University (2) 

North Carolina A&T University 
Purdue University 
Stanford University (2) 
University of Washington (10) 


Howard University (2) 
Stanford University (8) 
University of Minnesota (2) 
University of Washington (4) 


Case Western Reserve 
University 

George Mason University 
Howard University (2) 

John Brown University 
Lafayette College 

Michigan Technological 
University 

Minnesota State University, 
Mankato 

North Carolina A&T University 
Oklahoma State University 
Oregon Institute of Technology 
Purdue University 

Binghamton University, State 
University of New York 

Texas University State, San 
Marcos 

University of Colorado, Boulder 

University of Michigan at Ann 
Arbor 

University of Wisconsin, 
Madison 


Each Institute had a different theme (Table 5-B). The individual projects for the 2004-2005 
Institute were focused primarily on classroom changes under the broad theme of “classroom 
as lab.” The cohort drew from three CAEE partner campuses and two CAEE affiliates. A 
week-long Summer Summit was held on the University of Washington campus in July 2004 
that helped Scholars refine their research topics. The cycle concluded with a special session 
at the 2005 Frontiers in Education (FIE) conference in October (Adams et al. 2005). 

For the 2005-2006 Institute, Scholars worked on projects aimed at impact on engineering 
education at their campus (/.e., a theme of “campus as lab”). Nine Scholars from Stanford 
worked on one project together, and the remaining Scholars focused primarily on individual, 
campus-impact projects. The Institute began with a Summer Summit at Stanford University 
in June 2005. As part of the second Institute, the team created the IdeaLog (a wiki) to 
enhance communication among Scholars at remote institutions. 

The 2006-2007 theme was “Advancing Engineering Education Research to Meet the Needs 
of the 21st Century.” Scholars were recruited through a competitive, national application 
process and were asked explicitly to consider issues of diversity in their projects. As such, 
the focus was “nation as lab,” and participants considered change on a national scale and 
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collaborated to run a national workshop at the 2007 FIE conference in October (Ailendoerfer, 
Bates, etat. 2007). The 2006-2007 Scholars continued use of the IdeaLog deveioped in 
the previous Institute. 

Table 5-B summarizes representative themes and research topics for the three Institute 
cycles. 


Table 5-B: Themes and representative research topics for each ISEE cycle 


ISEE cycle 
year 

Theme 

Representative research topics 

2004-2005 

Class as lab 

Gender imbalance of students majoring in computer science 
Factors influencing student empowerment 
Acquisition of professional design expertise 

Creative thinking in 5th grade as compared to freshman 
engineering 

Student retention of materials sciences concepts overtime 
Attributes of workplace writing expectations 

Students’ understanding of life sciences and impact on choice of 
major 

2005-2006 

Campus as 
lab 

Student use of college of engineering resources 

Characteristics of early engineering education (e.g., fundamentals 
courses) and their impact on the student experience 

American and international students’ experiences in engineering 
majors 

Understanding the school-to-work transition 

Use of portfolios in teacher learning communities that seek to 
retain underrepresented minorities in engineering 

2006-2007 

Nation as lab 

Relationship of learning and personality styles with perceptions of 
success 

Factors influencing students’ global awareness; learning about 
cultural and global diversity 

Academic empowerment of Hispanic students transferring from 
community colleges 

Impact of case studies on students’ extrapolations to societal and 
global issues; presenting course material in socially relevant 
contexts 

Models for characterizing social justice in engineering 

Influence of cooperative education experiences on retention in 
engineering 
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5.2 The Impact of the Institutes 

The subsections below describe accomplishments and impact for the three Institute cycles. 
Subsection 5.2.1 describes the Influence of the year-long Institute experience on the 
Scholars, personally and professionally. Subsection 5.2.2 describes the Impact of the 
Scholars on engineering education, through their research projects and activities. 

In order to improve the overall Institute design, as well as to ascertain the Impact of the 
Institute on Scholars, a set of evaluation data was collected from Scholars Immediately 
before, during, and following each Summer Summit. There were also follow-up interviews 
with individual Scholars during the Institute year and a final, cumulative follow-up evaluation 
effort across all cohorts after the last Institute cycle. Data were gathered through surveys, 
focus groups, and Interviews. The results of this effort are discussed in Subsection 5.4. 

Many of the descriptions and quotations about impact included below are based on excerpts 
from the ISEE internal evaluation report produced by the University of Washington’s Office of 
Educational Assessment in April 2009 (Maring). 


5.2.1 The Impact of the Institute on Scholars 

This subsection draws on the cumulative evaluation findings to describe the impact on the 
Institute on the Scholars in the following areas: skills, knowledge, and experience; career; 
and community. The subsection concludes with a vignette (“Engineering an Interdisciplinary 
Career") that describes the effect on one Scholar In his own words. 

Skills, knowledge, and experience 

90% of all scholars reported In a follow-up survey that they learned more about educational 
research methods during the Institute than they had known prior to participation. When 
asked a separate question about the most important thing they learned, some of the most 
frequent responses included learning to do educational research and learning about specific 
methods or research techniques. In particular their familiarity with ethnographic observation, 
qualitative studies, quantitative studies, and teaching/design experiments. Scholars 
generally also indicated that they had increased their frequency of reading literature on 
engineering education and attending presentations on engineering education research and 
workshops on teaching in engineering. 


81% of survey respondents 
agreed or strongly agreed with 
the statement, “Being part of the 
institute was a positive addition 
to my CV. ” 


Career effects 

Evaluation data suggest that career effects were various, multi- 
faceted, and positive. These effects included an increased 
awareness of the field of engineering education research and 
participation in activities related to educational research. 

Scholars also felt that participation in the Institute had a 
positive effect on different types of career pathways and the 
reception of their work within their respective departments, 
colleges, and/or institutions. For example, 81% of survey 
respondents agreed or strongly agreed with the statement, 

“Being part of the Institute was a positive addition to my CV”; 71% agreed or strongly agreed 
with the statement, “The education research I did as an Institute Scholar was well-integrated 
into my career goals”; and 63% agreed or strongly agreed with the statement, “I believe that 
my participation in the Institute advanced my career.” 
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Tenured faculty saw the experience as a catalyst for incorporating engineering education in 
their research in a substantial way, and one intended to gain added legitimacy through 
external funding. Many of the early-career faculty saw ISEE as fitting perfectly within their 
current position, although some were less sure of the fit with their career progress. Graduate 
students noted that the Institute provided critical support to an interdisciplinary career in 
both engineering and education, helped prepare them fora teaching-oriented career, 
provided an opportunity to pursue research independent of their dissertation work, and 
opened up new career options (see Scholar focus in Figure 5.2-A). 

Those who found their institution supportive tended to investigate issues deemed important 
within their program, link efforts to ongoing evaluation needs such as ABET, and use the 
Institute to gain legitimacy for their work. 

The benefits of community 

Perhaps one of the most consistent findings across all four years of data about the Institute 
and impact on Scholars was how appreciative these individuals were to have been supported 
in the creation of a sustained community. When asked on the follow-up survey about the 
most important thing they learned, the most frequent response was some mention of 
community: the existence of it, the importance of having people available as resources, and 
of validation from being part of it. 

For some, it was this general sense of community— of realizing 
the existence of a group of like-minded individuals— that 
seemed to be an antidote to the sense of isolation within a 
department or institution reported by a substantial number of 
Scholars. In the follow-up survey, 84% of participants agreed 
or strongly agreed with the statement, “During my Institute 
experience, I made significant connections with other Institute 
Scholars.” For example, several 2006-2007 Scholars from 
different institutions, who had not known each other 
previously, went on to collaborate on research projects and/or 
grant proposals. 

Scholars’ comments suggest that their idea of ISEE community included members of the 
CAFE team working in the Institutes, and there was clear evidence that the ISEE team played 
a large role in ushering Scholars into the larger community of engineering education 
research. In the follow-up survey, the majority of responding Scholars indicated a sustained 
affiliation with the national engineering education community, with 65% agreeing or strongly 
agreeing with the statement, “Because of my Institute experience, I became more a part of 
the national engineering education community,” and 70% agreeing or strongly agreeing with 
the statement, “After my Institute experience, I remained a part of the national engineering 
education community.” 

Engineering an interdiscipiinary career 

The vignette in Figure 5.2-A describes the profound impact that ISEE had on one Scholar 
through his Institute research and interaction with other Scholars. 


84% of surveyed participants 
agreed or strongiy agreed with 
the statement, “During my 
Institute experience, I made 
significant connections with 
other Institute Scholars. ” 


128 


Enabling Engineering Student Success 


Figure 5.2-A: Vignette of a 2004-2005 Scholar and ISEE’s profound impact on his 
career (Maring, 2009) 


Scholar focus: Engineering an interdisciplinary career 

The Institute for Scholarship on Engineering Education was a perfect fit for one Scholar in the 
2004-2005 group, a graduate student in computer science and engineering who was seeking 
to study why so few women enter the fieid. Aiready determined to pursue this unconventionai 
topic for his dissertation, he was seeking guidance, training, and vaiidation that he couid not 
find within his department. 

According to this Schoiar, ISEE was a “big, formai and pubiic foray into engineering education 
research” and was an invaiuabie starting point for this Schoiar’s eariy career in engineering 
education. This inciuded the Summer Summit, when he found the community he had been 
seeking; continued with reguiar meetings among a smaii group of Schoiars and ISEE team 
members on his campus during the foiiowing year; and cuiminated with a poster at FIE in 
2005. In the fall of 2005, he described the impact of ISEE as follows: 

This project is reaiiy criticai to my remaining in the Ph.D. program, in that I had made a 
decision after finishing my Master’s work in traditionai computer science that I was 
much more interested in teaching, and if I couid compiete a dissertation in something 
cioser to education, then I couid see myseif compieting my Ph.D. 

After the Institute year, this scholar became a research assistant on the CAEE team working on 
the large-scale, longitudinal Academic Pathways Study (APS). In subsequent years, he 
continued to present at major national conferences, co-authoring papers with other CAEE team 
members and presenting his own work. He described the connection to CAEE as a vital way for 
him to get engineering education researchers interested in his work, as he describes below: 

/ had aii of these high-profiie opportunities where I wasn't [a research assistant] 
working on APS, but more pubiiciy estabiishing my identity as an engineering 
education researcher doing my own thing as weii. 

One example of such synergy was a workshop he developed with CAEE team members and an 
affiliate from the Women in Engineering ProActive Network (WEPAN). The interactive sessions 
(held at the national WEPAN and ASEE conferences in 2008) focused on several provocative 
quotes and findings from APS about how undergraduates perceive women in engineering. This 
Scholar helped develop the presentation and facilitate the conversations, and the workshops 
provided a perfect opportunity for him to discuss findings from his dissertation research, both 
formally and informally, with session attendees. 

Several years later, the impact of being an ISEE scholar and his connection to CAEE was clear. 
In 2008, he successfully defended his dissertation, with three CAEE team members serving on 
his committee and a former ISEE Scholar serving as his co-adviser. 

As this scholar put it. 

Whatever I end up doing, one thing is for sure, I don’t want to give up the momentum 
and the communities and the other great things I’ve gained whiie working at the 
Center and the Institute. ..[ISEE] is definiteiy one of the important pieces of my 
research career as a graduate student and, in particuiar, my engineering education 
research career. 
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5 . 2.2 The Impact of the Scholars on Engineering Education 

The ISEE Scholars demonstrated their ability to contribute to the scholarship of engineering 
education by their productivity regarding peer-reviewed publications, presentation of their 
work in different venues, and receiving awards drawing from their ISEE experience. 

During the course of the three Institutes from 2004 through September 2007, 20 Scholars 
had authored a total of 43 conference presentations or posters, and their production has 
continued in the ensuing years. Since the completion of the final Institute in 2007, a large 
majority of the scholars continue to be successful contributors to the community of 
engineering education researchers, with funding from the National Science Foundation (e.g.. 
Engineering Education Programs, Research and Evaluation on Education in Science and 
Engineering, Course, Curriculum, and Laboratory Improvement, etc.) and the Fulbright 
Scholars Program. Appendix A, Subsection A.4 lists selected Scholar outputs. 

The four vignettes in Figures 5-B through 5-E illustrate some of the impacts Scholars had on 
engineering education— both the practice of education and the foundation of knowledge on 
engineering teaching and learning. 
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Figure 5.2-B: Vignette of a 2004-2005 Scholar and his project on storytelling and 
learning of concepts (Maring, 2009) 


Project Focus: Scholarship of Teaching translates to industry training classes 

‘‘How does context increase student learning?” was how one Scholar (a self-described 
“dabbler” in the scholarship of teaching and learning) articulated his research question on the 
first pre-survey administered before the 2004 Summer Summit. 

During the week-long Summit, this Schoiar struggied to define and operationaiize the factors 
invoived in his question. In group discussions, he reveaied that he had been in a workshop on 
storyteiiing and frequency toid stories in his ciasses and wondered whether doing so affected 
student iearning. Throughout the week, he continued to specify aspects of his question: 
defining storyteiiing, understanding what aspects of student iearning he wanted to investigate, 
and expioring what methods he might use. By the end of the week, his research question had 
changed to "Do students deveiop deeper understanding when presented concepts through 
contextuaiiy reievant narratives in the format of ten-minute stories?” 

With such a specific focus, this schoiar had a strong foundation for foiiowing through with his 
project. After heip from the ISEE team with his human subjects research appiication, he 
coiiected both survey and interview data about the impact of his stories on student iearning. At 
the foiiow-up interview in the summer of 2005, preiiminary resuits suggested that the 
storyteiiing was effective. More importantiy, the process of working on the research project 
had a strong impact on his ciassroom practices. During the year, he had deveioped additionai 
stories to use in his ciassroom and had appiied a more structured, systematic, 10-minute 
format (which he had iearned at the workshop) to stories he aiready toid. 

In 2006, this Schoiar ieft his university facuity position and started working in industry. The 
evaiuator was abie to foiiow up with him, and, in 2007, he indicated that he was no ionger 
using his ISEE project research as part of his professionai work. However, in 2008, the Schoiar 
changed his industry position and began teaching a course on engineering basics to empioyees 
at the company where he worked. He indicated that he now did more teaching than when he 
was at a university. The ciass was originaiiy pianned for 30 peopie, but it turned out to be in 
such high demand that, by June of 2008, he had aiready taught the ciass three times within a 
six-month period to 300 empioyees. He noted that the storyteiiing techniques he had refined 
and studied as an ISEE scholar were a prominent part of these popular training sessions. 
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Figure 5.2-C: Vignette of a 2004-2005 Scholar and her continuing involvement in 
engineering education research (Maring, 2009) 


Scholar focus: Progress amid resistance 

One Scholar from the first cohort continued involvement in engineering education research and 
provided reflections on her work at severai points between 2004 and 2008. In general, the 
theme of these reflections was that she had experienced resistance from her department and 
felt isolated as she pursued this line of work. Remarkably, she had also made considerable 
progress, not only in generating research in the area, but also in developing an innovative 
course for students that has become institutionalized. 

This Scholar began her work driven to understand undergraduates’ experiences in engineering 
and in particular in the effects of empowerment on students’ performance. She teamed with a 
graduate student who was also an ISEE Scholar, and, in the first Institute year, collected survey 
data from students at three institutions across the country, focusing on differences in the 
student experience between men and women, as well as across different regions in the 
country. 

Another graduate student took on the project and became an ISEE Scholar in the 2005-2006 
cohort. At this point, the project shifted due to preliminary findings to focus more on the extent 
to which students had a sense of belonging in engineering. The two Scholars collected data 
from students, as well as from practicing engineers, about belongingness. In addition, they 
developed a syllabus for a “savvy sessions” course to support undergraduates in navigating 
their undergraduate years and foster a sense of belonging. As the Scholar put it, “Increasing 
belonging in any environment tends to enable minorities and under-represented populations to 
persist.” 

These “savvy sessions” were piloted during the following academic year. Although this Scholar 
was no longer working with research assistants through ISEE, she continued developing this 
course, sought and received external funding for related research, and entered a Masters in 
Education program with a focus on cognitive studies. In reflecting on her decision to obtain an 
additional degree, the Scholar stated simply, “I thought if I was going to do research in 
education long-term, surely I should train myself appropriately.” 

When the evaluator spoke with this Scholar in 2008, she reported that she had taught a 
revised version of the “savvy sessions” twice, despite some opposition from her colleagues. 

She indicated that she overcame the strong resistance when an ABET review indicated that the 
College of Engineering was missing courses in ethics and broader context. After making some 
revisions to her course, she was able to incorporate these ABET components, while also 
fostering a sense of belonging among students 

This Scholar reported that not only has her course become an official part of the curriculum in 
engineering, but that it could potentially become a required component of the undergraduate 
degree program. This Scholar has also recently received an NSF grant that builds on these 
themes of belonging, community, and engagement. 

However, when reflecting on the impact of the Institute experience on her career, the Scholar 
stated, 

/ don’t have a fairy-tale story. The technical area of my research goes up and down 
quite a bit, so I need to retrain, and what better area to retrain than in something that 
Tm interested in. It’s an obvious choice. ..My only regret is that [with federal funding], 
you get all this marvelous support and buoyancy and help getting started and then you 
get left on your own. ..I would prefer a collaborative work environment. 
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Figure 5.2-D: Vignette of a group of 2005-2006 Scholars and their large-scale 
project (Maring, 2009) 


Project focus: Raising questions about engineering curricuium on campus 

During the second Institute, a group of Scholars from one institution worked together on a 
large-scale project with the purpose of effecting change in engineering education on their 
campus. One of the Institute leads assembled a team of Scholars consisting of engineering 
graduate students with an interest in educational research, professional staff working in the 
university’s Center for Teaching and Learning (CTL), and faculty who served on the 
undergraduate council (a body that makes decisions about engineering curriculum). 

During the Summer Summit and after much discussion, the group settled on a set of classes 
referred to as “engineering fundamentals,” with a goal of answering the two following 
questions: 

1. What do faculty who teach engineering fundamentals courses think about them: Do 
they serve their originally intended functions and what is it like to teach these 
courses? 

2. Do the fundamentals courses fit within the curriculum in a way that aligns with the 
stated goals to provide a broad introduction to engineering concepts and expose 
students to different areas of engineering? 

During the course of the following academic year, the team met fairly regularly, and, most 
importantly, two people Joined the team. The first was the Associate Dean for Student Affairs in 
engineering and the second was a member of the CAEE team who came on board as a project 
manager. The graduate students, along with the project manager, gathered transcript data to 
try and answer the second question (above), while staff from CTL, in collaboration with other 
Scholars on the team, set up a series of faculty luncheons for instructors of engineering 
fundamentals classes to address the first question (above). 

One year after the Summer Summit, members of the team considered the project a success. 
They had analyzed relevant data, answered their two questions, and already seen some impact 
of the work. Most immediate was a notable increase in the amount of discussion and 
reflection about these courses. The faculty luncheons provided a forum for the instructors to 
discuss their experiences, and two participants later consulted with CTL to discuss how to 
incorporate learning goals into their classes. These instructors’ conversations with each other 
also created a sense of community where there had been very little connection up to that point. 

Subsequently, the project continued with the Associate Dean taking the lead in trying to use 
the findings to stimulate additional curriculum change. The team presented at a meeting of all 
deans and department chairs in engineering, and CTL continued to work with engineering 
faculty on the fundamentals courses and on teaching issues in general. 

Two years after the project (during the 2007-2008 academic year), the Associate Dean funded 
one of the graduate students and the project manager part-time to collect data on other 
institutions for the purpose of benchmarking engineering fundamentals courses. A series of 
reports were generated and submitted to the Associate Dean for future use. In the fall of 
2008, the project manager commented, 

...the Dean of the School of Engineering is publicly talking about the notion of design 
thinking as an engineering fundamental. I don't know that this is necessarily an 
outcome of our work.. .but the role and importance of design in the early engineering 
experience is something we had talked about in our early ISEE conversations. 
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Figure 5.2-E: Vignette of a 2006-2007 Scholar’s extensive STEM education work 
during and beyond her institute year (Maring, 2009) 


Project focus: Integrating engineering into middle school curriculum (ISEE Scholar 2007) 

“Well, my kids are in middle school,” one Scholar reported, when asked why she decided to 
add work on engineering education to an already-thriving career in chemical engineering. On 
her pre-survey, the Scholar indicated an intention to study the perceptions of engineering 
among middie schooi teachers. By the end of the Summer Summit, she had come to re-think 
and re-scope her project, reaiizing it wouid occur in three phases: first understanding teachers’ 
perceptions of engineering education, then comparing these to teacher candidates’, and finaiiy 
determining ways that engineering concepts couid be integrated into middie schooi curricuium. 

One year after the Summer Summit, this Schoiar had four papers at the 2007 ASEE 
Conference, one based on her ISEE project. As she told the evaluator, "Well, remember how I 
thought my one year project was a three-year project? I am doing all three things at the same 
time.” She was in the process of analyzing data on both pre-service and in-service teachers 
about their perceptions of engineering and had developed both an engineering camp for 
middle school students and a mentoring program for female students interested in math and 
science. She had also worked with teachers to integrate engineering concepts, not Just into 
math and science, but also into social science and English classes. 

In addition, this Scholar had reached beyond her original proposed project to consider other 
questions. In her work with middle schools, she targeted one particular rural county with a 
significant Native American population and had become particularly interested in bringing 
engineering concepts to schools in that region. She was also in the process of writing an NSF 
Research Experience in Teaching grant for teachers in that county. She continued to present 
the results of her work, with three papers at the 2007 FIE Conference. 

At ASEE 2008, this Scholar co-authored a paper with an education graduate student whom she 
was mentoring. This student was conducting a study on how engineering faculty started to 
engage in scholarly work on teaching and learning. The Scholar indicated that she was still 
extremely busy in research on STEM education, conducting workshops for math and science 
teachers, working under a CCLI grant on fostering critical and creative thinking in freshmen, 
and starting a study on how math is (or is not) contextualized for freshman seeking to study 
engineering. 

She Jokingly reported on the impact of the Institute by saying, "Well, I am really busy, and it’s all 
[the ISEE lead’s] fault! I wake up in the middle of the night thinking about new research 
questions!” 
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5.3 The ISEE Model 


The ISEE model (Adams et a/. 2006) includes content and activities that focus on building 
community and developing an understanding of how to design and conduct engineering 
education research (similar to other capacity-building programs). The model also draws on 
previous efforts to target particular challenges of engaging in this kind of work. Unique 
attributes of the ISEE model include the use of research-story posters and storytelling; 
development of plans to enhance the scholarship of impact; approaches to studying 
diversity; and approaching educational experiences (at the class, campus, or national level) 
as laboratories for investigating student experiences and learning. 


The paragraphs below provide an overview of the Institute year 
and an accompanying set of key strategies and design 
principles. These highlights draw from the more detailed 
discussion of the ISEE model in Adams etal. 2006. Additional 
details of ISEE implementation are provided in Allendoerfer, 
Bates, et al. 2007; Allendoerfer, Jones, et al. 2007; and Lande 
etal. 2007. 

5.3.1 Five Stages of ISEE 

The Institute year consists of five stages, as described below: 


Unique attributes of the ISEE 
model include the use of 
research-story posters and 
storytelling, and approaching 
educational experiences as 
laboratories. 


Recruiting and selecting Scholars 

Each ISEE cycle brings together a cohort of engineering faculty, graduate students, and (in 
some cycles) educational support staff. The process of reviewing and selecting candidates 
emphasizes recruitment of diverse participants, with a focus on building community. An 
important consideration in assembling the cohort is to promote change at host and affiliate 
schools/institutions. Honoraria for faculty and fellowships for graduate students are 
provided. 


The Summer Summit 

This week-long, intensive, interactive, face-to-face learning experience launches the Institute 
year. During the Summit, Scholars (1) learn about research design and methods from the 
learning sciences; (2) have opportunities to practice research methods; (3) develop as a 
community; (4) discuss current issues in engineering teaching and learning as part of 
deciding on a research topic; and (5) formally define a research study to be conducted during 
the academic year. 


Conducting a year-long research study 

During the academic year. Scholars are mentored (by both peers and experts) as they 
implement their research study. A variety of methods are used to sustain community and 
provide resources for moving studies forward (e.g., just-in-time presentation of content, work- 
in-progress meetings, invited experts across the community). A collaboratively maintained 
web site or “wiki” called the IdeaLog was developed for use in the second and third Institutes 
to build and support a community of practice among the Scholars. This online tooi was used 
both during the Summer Summit and while the Scholars were on their home campuses. The 
IdeaLog is like an informal sketchbook in which Scholars have a shared space to capture 
information and inspiration in ways that are personally meaningful. To promote continual 
feedback. Scholars are encouraged to visit and respond to each other’s postings. The tooi 
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was initially implemented with an understanding that the IdeaLog requires adaptation and 
tuning to the adopting community (Chen et at. 2005) 

A culminating Leadership Summit 

This is an event in which Scholars present their work to specific communities (e.g., at 
professional conferences or local meetings), are introduced into the broader engineering 
education community, and refine leadership skills. In the first and third of the three 
Institutes sponsored by CAEE, the culminating summit took the form of an interactive session 
at the Frontiers in Education Conference (Adams etal. 2005; Allendoerfer, Bates, etal. 

2007). 

Mapping and adapting the Institute model 

In this iterative design stage, the ISEE leadership team collaborated with an evaluation team 
from the Office of Educational Assessment at the University of Washington to distill what had 
been learned from the previous Institute. The team identified opportunities for improvement 
and aligned the Institute model with the needs of the next host campus. 

Formative and summative evaluation played crucial roles in this iterative process. 

Information from surveys of Scholars before their initial Summer Summit meeting was used 
to tailor Summit activities. Focus groups with Scholars informed improvements to the 
IdeaLog and to community-building activities during the academic year. Evaluative interviews 
with Scholars provided additional insight and advice into the challenges of becoming an 
engineering education researcher. 

A sample schedule for the week-long Summer Summit is shown in Table 5-C. 


Table 5-C: Sample schedule for an ISEE Summer Summit 


Day 1 

Day 2 

Days 

Day 4 

Day 5 

Institute Theme 

Defining a Good 

Research 

Question 

Draft 

Presentations of 
Research Plans 

Critiquing 

Research 

Study 

Presentations 

Motivations for 
Research 

Research Tools 

Draft 

Presentations of 
Research Plans 
(cont.) 

Making Sense 
of Research 
Frameworks and 
Theories 

Study 

Presentations 
(cont.) and 
Reflection on 
Being a 
Community 

Lunch 

Lunch 

Lunch 

Lunch 

Lunch 

Posters and 
Group Work 

Research Tools 
(cont.) 

Rigor, Sampling, 
and Bias 

Individual Work 
Time 

Next Steps 

Communication 
and Community 
Building 
Technology 

Study Design 

End-of-Day 
Social and 
Evaluation 

Designing 
Studies for 
Impact and IRB 
Issues 

Evaluation Focus 
Groups 

End of Day 
Social and 
Evaluation 

End of Day 
Social and 
Evaluation 

Homework 

End of Day 
Social and 
Evaluation 

End of Summit 

Homework 

Homework 

Social Time 

Homework 
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5.3.2 Strategies to Help Scholars Succeed 

We found that Institute Scholars who were new to the field of educational research were 
greatly helped by participating in a local community of like-minded colleagues. This was 
particularly important for giving and receiving feedback about research work in progress. 

We also found that the engineering faculty were often unfamiliar with particular aspects of 
educational research, such as navigating a new disciplinary language, utilizing qualitative 
research methods, and/or analyzing qualitative data. We developed a range of strategies 
that were useful in helping Scholars move past these barriers: 

• providing structured, formal interactions during the Institute year; 

• using project plans as milestones; 

• providingjust-in-time activities that introduce various research methods and 
educational theories; 

• providing templates and guidelines for human subjects processes; 

• using work-in-progress meetings to promote accountability and support interactions 
among the Scholars and ISEE team members; 

• seeking campus-based resources (and working with local administrators to access 
these resources); 

• continuing efforts to build community (outside ISEE in particular) as a means to 
support motivation; and 

• implementing activities about how to build social networks and how to communicate 
with different audiences. 

5.3.3 ISEE Design Principles 

During the course of multiple iterations of the Institutes, the team developed a set of 
overarching design principles behind the ISEE model. These include 

• using a tiered waterfall model for recruiting Scholar cohorts and building local 
community over time; 

• adopting a user-centered approach (informed by evaluation activity conducted 
throughout the ISEE process) to identify and address the challenges Scholars 
encounter before or during their ISEE experience; 

• creating a focus on learning contexts as “laboratories for investigation and impact,” 
where Scholars have or can create the influence to enact change; 

• engaging in interactive, community-centered activities (e.g., interactive study posters, 
work-in-progress sessions, and sharing ISEE stories); 

• establishing a safe environment for open discussion and feedback; 

• using both structured and unstructured/informal activities for building community; 

• having the Scholars complete the first phase of activity with at least a draft study plan; 
and 

• recognizing that a focus on impact is a key motivation for participation in this 
community. 
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Fundamental to all of these principles is the understanding that the ISEE experience is about 
facilitating relationships that encourage shared construction of knowledge, mentoring, and 
enabling others to pursue a passion. 

5.3.4 Additional Implementations and Adaptations 

Aspects of the ISEE model have been adapted and expanded by others in the engineering 
education research community. These include the use of Scholars’ research posters 
developed during the summer summit; expansion of curricular materials and scholarship 
models to an international audience; use of findings from the Institute cycles in the 
development of a Ph.D. program in Engineering Education at Purdue University; and a next- 
generation partnership with the NSF-funded Rigorous Research in Engineering Education 
(RREE) program for building capacity in engineering and engineering technology education 
(Streveler and Smith 2006). 


5.4 Research Findings on How Educators Enter, Navigate, and Work 
in the Interdisciplinary Field of Engineering Education 

As part of the overall goal of building capacity in engineering education research, in 2006- 
2007, the ISEE team investigated details of the pathways into engineering education 
research, examining how people enter, navigate, and work in this type of interdisciplinary 
space. An additional goal of this study was to provide feedback into the ISEE model 
developed by the team (as well as other capacity-building programs) regarding factors that 
can enable engineering education research pathways and the construction of identities. 

This study of pathways included 13 engineering educators 
selected to represent varying levels of membership in the 
engineering education research community (e.g., low to high 
level of contributions, low to high level of formal learning in 
education research). Semi-structured interviews were used to 
engage participants in conversations about their engineering 
education work, how they began doing engineering education 
research, what challenges they faced and strategies for 
overcoming those challenges, and what advice they would give 
to others who wish to enter the community. (Allendoerfer, 

Adams, et al. 2007) 

Analysis of the interview data suggests common elements in people’s pathways into 
engineering education research, although there are also many unique characteristics. Two 
significant themes that emerged, among a number of others, are 

• the importance of a community of practice perspective and 

• the development of a composite identity, with many frames and many combinations 
that help to establish an identity as a cross-disciplinary engineering education 
researcher. 

5.4.1 A Community of Practice Perspective 

The community of practice perspective involved exploring the data through interrelated 
lenses of domain, community, and practice. Here, domain is marked by a commitment to a 
shared domain of interest, community by engaging in joint activities and relationship 


This study of pathways included 
13 engineering educators 
selected to represent varying 
levels of membership in the 
engineering education research 
community. 
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building, and practice by a shared repertoire of resources, experiences, tools, and ways of 
addressing recurring problems. Participation in a community of practice with respect to 
engineering education research allowed the participants to engage in joint activities and 
discussions, help each other, share Information, and build information that enabled them to 
learn from each other. 


The study participants shared a “domain of interest” that implies a commitment to the 
domain and shared competence that distinguishes members of the domain. Participants 
shared a common entry point into engineering education research that included experiences 
with teaching and learning that generated researchable questions. Participants also 
commonly focused on application-oriented research, though some were oriented toward 
basic research. They all shared a passion for supporting student learning and enabling 
effective learning environments. Further, the participants demonstrated their commitment 
to the domain by pursuing formal knowledge, either through education or collaborations. 


Participants engaged in the community of engineering 
education research to varying degrees through small and local 
community entry points; other people served as catalysts Into 
the domain or as motivation for continued commitment. 

Community building practices included using informal 
networks as pathways to the broader community, networking 
strategies that may be characterized as “intentional 
serendipity” rather than “luck,” proactively building 
collaborations that enable shared learning and sustaining 
passion, and interacting in a variety of communities that can 
support multiple identities. Community played a pivotal role In 

the development of an engineering education research identity and was observed to interact 
substantially with the themes of domain and practice. 


Community building practices 
included networking strategies 
that may be characterized as 
“intentional serendipity. ” 


A number of engineering education research practices were observed in the evaluation 
interview data. These include determining where to find Information and how to decide if it is 
useful, experiencing different forms of inquiry, seeking rigor, and coming to terms with the 
complexities of studying humans. There were also community-building practices to create 
collaborations that help address Inadequacies in training or knowledge and that bring new 
people into the practice. Cross-disciplinary practices included activities to bridge 
perspectives and cross-disciplinary tendencies and identities. Institutional and cultural 
practices Included “spinning” stories for different audiences and participating in traditional 
academic culture with “non-traditional” work. 


5.4.2 Developing a Composite Identity 

In addition to finding ways to Join the community of engineering education research at both 
local and broader levels, participants reported developing a composite identity as their new 
involvement in this community expanded. This composite identity was made up of many 
combinations for making sense of self. The frames associated with this composite identity 
included “interdisciplinary,” teacher, agent of service, educational researcher, and 
engineering researcher. Participants were essentially “performing an identity”: attending to 
situational aspects of their careers and lives by strategizing an Identity In relation to an 
audience. For example, participants used an identity of “teacher” to engage others in 
conversations about student learning, “educational researcher” to participate In the broader 
educational research community, and “Interdisciplinary” to act as a bridge between 
traditional engineering faculty and the community of engineering education research. 
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5.5 Summary 


The three cycles of the Institute for Scholarship on Engineering Education (ISEE) component 
of CAEE resulted in significant capacity-building in engineering education research, both in 
terms of people and program models. First, it helped a total of 49 faculty, graduate 
students, and staff representing 20 institutions develop as new engineering education 
researchers. As Institute Scholars, they were guided through 
the process of designing and carrying out engineering 
education research projects. Second, ISEE produced and 
refined a model describing the content and activities used to 
prepare new engineering education researchers, with an 
emphasis on buiiding community, within each cohort of 
Scholars, as well as the larger engineering education 
community. This model includes a set of specific design 
principles, including those that highiight techniques for 
recruiting prospective Scholars and the importance of 
interactive, community-centered activities. 


/SEE helped a total of 49 faculty, 
graduate students, and staff 
representing 20 institutions 
develop as new engineering 
education researchers. 


Evaiuation data describes the multiple ways in which Institute participation impacted the 
Schoiars— in terms of skiiis, knowiedge, and experience; career paths; and community. At the 
same time, the Scholars themselves, through their research projects and associated 
activities, impacted engineering education in a variety of concrete ways. Exampies inciude 
innovations in industry training and integration of engineering across the middle school 
curriculum in a rurai county with a significant Native American population. 

Other contributions of this work inciude identifying strategies that have proved useful in 
heiping Scholars face common challenges in undertaking engineering education research. 
These strategies address a wide range of activities, from handiing the human subjects 
research approvai process to integrating with the engineering education community. Finally, 
a research study of 13 engineering education researchers detailed two significant aspects of 
their pathways into the fieid of engineering education research: the importance of a 
community of practice perspective and the development of composite identity. 
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6 Getting the Word Out 


The CAEE team made significant efforts to get the word out about the work of the Center. A 
fundamental part of this activity was sharing news about the research and results with as 
wide a variety of audiences and at as many different venues as possible. These 
dissemination activities began early in the life of the Center and continue beyond the formal 
end of the grant period. 

This section summarizes some of the methods we have used to let others know of our 
research. In addition to publications and presentations, we list examples of the research 
Instruments and other materials made available for use by others, of programs developed for 
use and adaptation, and of the impact that the many people affiliated with the Center, either 
as team members or collaborators, have had in spreading the word. 


6.1 Sharing Findings at Different Venues, 

Addressing Different Audiences 

The large and varied body of data produced by CAEE provided the foundation for over 100 
scholarly publications. In addition to engineering education publications, the team sought to 
disseminate findings through presentations, special sessions, and workshops. 

Major activity from January 2003 to June 2010 included 

• over 130 papers and Journal articles In both engineering education and education 
publications, including 6 articles In the July 2008 special issue of the Journal of 
Engineering Education] 

• 9 plenary and keynote presentations; and 

• 9 conference special sessions. 

Publication activity spanned Journals and proceedings that focused on engineering 
education, education, the learning sciences, sociolinguistics, and anthropology. 

As a part of building a national presence, CAEE team members presented findings in invited 
talks, keynotes, and plenary addresses. Examples (with venue, date, and title of the talk) 
Include the following: 

• Frontiers in Education (FIE) Conference (2009): “Educating the Well-Rounded 
Engineer: Insights from the Academic Pathways Study" 

• Women In Engineering ProActive Network (WEPAN) National Conference (2009): 
“Women In Engineering: Interests, Perspectives, Confidence. ..and Experiences” 

• Rice University Dean’s Invited Lecture (2008): “Does Teaching Matter? (Making a 
Case for Better Teaching)” 

• NSF Science, Technology, Engineering, and Mathematics Talent Expansion Program 
(STEP) Meeting (2008): “Undergraduate Pathways to Engineering: Insights from a 
Longitudinal Study" 
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• ASEE Global Colloquium on Engineering Education (2008): “Implications of the 
Academic Pathways Study (APS)” 

• Center for the Advancement of Scholarship in Engineering Education (CASEE) Annual 
Meeting, Dane and Louise Miller Symposium (2007): “CAEE: An Overview of 
Accomplishments to Date” 

• Danish Centre for Engineering Education Research and Development, opening 
address (2007): “Engineering Education Research: Some History and Examples from 
the U.S.” 

• American Society of Mechanical Engineers Chairs Meeting Plenary Session (2007): 
“Center for the Advancement of Engineering Education (CAEE): Research on 
Engineering Education” 

Examples of national conference special sessions dedicated to CAEE research include the 
following: 

• FIE Conference (2009): “Research Findings on Engineering Student Learning and 
Engineering Teaching: Interactively Exploring the Implications for Engineering 
Education” 

• ASEE Annual Conference (2009): “Findings from the Academic Pathways Study of 
Engineering Undergraduates 2003-2008, Overview and Panel Discussion” 

• ASEE Annual Conference (2008): “Describing the Engineering Student Learning 
Experience Based on CAEE Findings,” Part 1 and Part 2 

• FIE Conference (2007) Communities of Practice in Engineering Education: “How Do 
We Investigate Diversity and Global Engineering?” 

• American Educational Research Association Annual Meeting (2006), Round Table: 
“Identity in the Making: Engineering Students’ Pathways to Their Professions” 

• FIE Conference (2005): “Communities of Practice in Engineering Education: What Are 
We Learning?” 

In addition, the team offered more than 25 workshops based on CAEE findings to groups at 
the WEPAN, FIE, Professional and Organizational Development Network in Higher Education 
(POD), and CASEE national meetings, on local CAEE campuses for faculty and dean’s office 
staff, and at other many local and national venues. 

The team also took advantage of opportunities to present CAEE findings to audiences 
outside of the traditional scholarly publications and conferences in engineering education. 
Highlights of some of these include the following: 

• ASEE Prism teaching toolbox article (January 2010): “Not What Students Need” 

• Anita Borg Institute for Women and Technology: Gendered Pathways to Success in 
Engineering (November 2007): “Results from CAEE’s Academic Pathways Study” 

• U.S. House of Representatives STEM Education subcommittee caucus (November 
2007): “Design and Student Engagement: Some Differences by Gender” 

• Alliance for Graduate Education and the Professoriate (AGEP) Capacity-Building 
Meeting (February 2007): “CAEE - Implications for your Retention Study” 
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Several final examples describe the use of CAEE’s findings to address existing problems 
and/or effect change: 

• All five of the campuses involved in the Academic Pathways Study research used 
results from the study to address specific questions on their campuses with respect to 
programmatic changes, potential curricular reform, illumination of student responses 
on another national survey, or to better ascertain students’ thoughts on a variety of 
topics. 

• Results from one of the workplace studies were used to engage managers at a 
participating company in making changes to their on-boarding process for new hires. 

A complete listing of publications, presentations, special sessions, workshops, and other 
dissemination activity is available in Appendix A of this report and on the CAEE web site. 


6.2 Creating a Set of Resources for General Use 

One very important outcome of the development of the various research studies was the 
creation of materials that could be used by others. These include a set of resources that are 
available to assist others in conducting similar research: the PIE and APPLE surveys; the APS 
structured, ethnographic, and workplace interview protocols; the engineering design task 
instruments; and the SEED interview protocol. In order to provide a detailed look at the 
design and implementation of the Academic Pathways Study, we created the summary 
document, “An Overview of APS Research Processes & Procedures” (Sheppard etal. 2009). 

The team also created several sets of program materials that can be used as models and/or 
guides by those interested in conducting similar programs. These include the ETPP 
curriculum and supplemental materials and the ISEE Summer Summit outline. The ASEE 
2006 paper that describes the ISEE models also provides information for those interested in 
staging similar capacity-building events (Adams et al. 2006). The record of web site visits 
and downloads suggests that these tools and materials have already been used extensively 
by others. 

As a way to provide a quick introduction to 
CAEE’s research, the team created a series of 
research briefs that summarize findings and 
implications from published papers in short, 
easy-to-scan documents. The briefs allow 
interested readers to rapidly get a sense of the 
scope and content of our research. Over 60 
articles and papers are summarized in the 
briefs to date. 

These materials and resources (described more fully below) are all available on the CAEE 
web site, which provides additional background and history for the Center. The web site has 
averaged over 1,000 visits per month over the last several years. 


Materials are available on the CAEE 
web site; 

http://www.engr.washington.edu/caee 
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6.2.1 The Academic Pathways Study (APS): Detailed Study Description and instruments 

The APS team created a detailed description of how the Academic Pathways Study research 
was designed and implemented, from initial conception through administration of the APPLE 
Survey in 2008. This technical report is available on the CAEE web site under the following 
title: 

• CAEE-TR-09-03: An Overview of the Academic Pathways Study: Research Processes 
and Procedures 

Of particular interest are the specific instruments and protocols for the different methods 
and cohorts that are included as appendices in the technical report, as listed below: 

• Structured Interview protocols 

• Engineering Design Task protocols 

• Ethnographic Interview protocols 

• Workplace Interview Guide: Big car company 

• Workplace Interview Guide: County-State-Aerospace 

• Persistence in Engineering (PIE) Survey instrument 

• APPLE Survey Questions 

6.2.2 Studies of Engineering Educator Decisions (SEED): interview Protocois 

The SEED interviews used a Critical Decision Method approach, in which engineering faculty 
were asked to identify critical incidents in their decision-making process. Interviewees were 
asked to explain their teaching decisions in general and to identify two recent and 
memorable decisions: a planning decision made in advance of a class or other interaction 
with students and a more immediate, interactive decision made “in the moment.” 

• SEED Interview Protocol 

6.2.3 CAEE Research Briefs 

The CAEE research briefs were created to summarize research findings in short, easy-to-scan 
documents. Over 60 articles and papers are summarized in the briefs. The research briefs 
are accessible through a page on the CAEE web site that groups the briefs under the 
following five subject themes: 

• Understanding Student Pathways and Experiences 

• Examining Student Learning and Skill Development 

• Exploring Issues of Diversity 

• Developing Community and Building Models for Engineering Education Research 

• Developing Effective Teaching Practices 

The Research Briefs are also linked from individual papers and articles listed on the 
Publications page of the CAEE web site. 
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6.2.4 Program Materials: ETPP and ISEE 

Two components of CAEE’s activity (ETPP and ISEE) were focused on creating and 
impiementing programs that targeted different populations but had a simiiar goal to build 
community. Both programs were successfuily demonstrated by the CAEE team, and each 
program served as a powerfui means of reaching out to other audiences. Each program was 
adopted or adapted by others. We summarize the available materials below. For more 
complete descriptions of the two programs, see Section 4 (ETPP) and Section 5 (ISEE) of this 
report. 

The Engineering Teaching Portfoiio Program (ETPP) was focused on building the skiiis of 
engineering graduate students interested in careers in teaching. Curriculum and 
supplemental materials are available through the CAEE web site. 

The ETPP curricuium describes a program that uses the development of a teaching portfolio 
as a means to expand the skiiis of engineering graduate students and recent Ph.D. 
graduates who are interested in teaching. The curriculum serves as a guide for the eight 
peer-facilitated sessions during which participants create and discuss teaching and diversity 
statements and provide annotations of representative teaching artifacts. Suppiemental 
materials used in the originai offerings (2004-2005) provide examples and background 
information. 

• ETPP Curriculum 

• ETPP Supplemental Materials 

The Institute for Scholarship on Engineering Education (ISEE) was focused on expanding the 
community of engineering education researchers. An expanded description of the 
development of the Institute modeis (with a focus on establishing a community of practice) 
was presented at the 2006 ASEE Conference (Adams et al. 2006). The representative 
Institute Summer Summit Schedule provides an example of the organization and topics 
covered during the five-day summit meeting that began each year-iong Institute cycle. The 
following ISEE materials are available through the CAEE web site: 

• Institute Model (Adams et al. 2006) 

• Institute Schedule 


6.3 Getting the Word Out... Through a Large and Diverse Team and 
Collaborating with Others 

6.3.1 The CAEE Team 

The CAEE team began as a group of researchers from five universities representing eight 
disciplines in engineering and education. An explicit goal in assembiing the team was to 
combine researchers from both engineering and education departments who had a mix of 
quantitative and quaiitative research expertise. 

Over the course of the grant, the Center grew to involve 63 faculty members and staff, 41 
graduate students, and almost 50 undergraduates during the period 2003-2010. As 
research scientists and graduating Ph.D. students took positions at other campuses, they 
typicaiiy continued their involvement with CAEE, spreading the Center’s infiuence even 
further. 
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Figure 6.3-A: The growth of the CAEE team and affiliates (2003-2009) (Maring 2009) 
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Figure 6.3-A shows sequentially the expansion of the CAEE team from the beginning in 
January 2003 through March 2009 (Maring 2009). The maps present a cumulative record 
of the growth of the team, movement to other campuses, and coilaborations. The figures 
aiso inciude extended team members: the Institute Scholars and Advisory Board members. 

Much of the movement to other campuses shown in Figure 6.3-A resulted from the growth of 
CAEE team members’ careers. Research scientists and graduate students were “moving up” 
to faculty positions as assistant professors: 

• Two Joined Purdue University’s recently established School of Engineering Education. 

• One went to an education faculty position at the University of Rochester. 

• Another Joined the Department of Psychology at the University of the Virgin Islands. 
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• One became a member of Olin College of Engineering’s Department of Mechanical 
Engineering. 

• A former graduate student began a position in the College of Education at the 
University of Illinois. 

• Another CAEE graduate researcher went to Virginia Tech’s Department of Engineering 
Education. 

• A third graduate student joined the University of Portland Computer Science faculty. 

In addition, three graduate research assistants used CAEE research as the basis for their 
Ph.D. dissertations. 

6.3.2 Collaborating Beyond the Center 

From the beginning in 2CC3, the CAEE team sought to engage with colleagues at other 
national research and educational organizations, as well as a selection of companies with an 
interest in the education of future engineers. The team’s goal in this was two-fold: to 
increase the audience for the growing body of our findings and to help others in their efforts 
to improve engineering education, whether through using CAEE’s results or through using 
CAFE’S tools and methods to conduct their own research. Some highlights are provided 
below. 

Women in Engineering ProActive Network (WEPAN) 

CAEE began contact with WEPAN early in 2CC3 through Sherry Woods, at that time president 
of WEPAN, who was also a member of the CAEE Advisory Board. Extending beyond her role 
on the Board, she was an active participant in annual CAEE team meetings and Academic 
Pathways Study data workshops, providing insight based on her expertise on the role of 
women in engineering. With her assistance, CAEE team members presented a workshop and 
the keynote address at the WEPAN National Conferences in 2008 and 2009, respectively. 

Center for the Advancement of Scholarship on Engineering Education (CASEE) 

Norman Fortenberry, Director of CASEE, served as an Advisory Board member over the last 
two years of activity for the Board (2006-2007). For the 2007 CASEE Annual meeting, CAEE 
Director Cindy Atman was invited to give the plenary address for the Dane and Mary Louise 
Miller Symposium, and she and other CAEE team members also presented an interactive 
workshop that engaged the attendees in discussion of APS results. 

Center for the integration of Research, Teaching, and Learning (CIRTL) 

CIRTL and CAEE were the two centers funded by NSF in 2003 as higher education Centers 
for Teaching and Learning. CIRTL Joined CAEE in sharing a booth at the ASEE Annual 
Conference in 2003 and 2004. CIRTL researcher Sandy Courter was there both years and in 
2004 she was accompanied by graduate students Alice Pawley and Gina Svarovsky. CAEE 
team members also participated in all three CIRTL Forums in 2003, 2005, and 2008. 

National Action Council for Minorities in Engineering (NACME) 

CAEE’s initial contact with NACME was in 2004 when Daryl Chubin, Senior Vice President for 
Policy and Research, reviewed an early draft of the Persistence in Engineering (PIE) survey. 
Four CAEE team members also met with Dr. Chubin to discuss diversity in engineering and 
possible dissemination strategies for CAEE results. CAEE’s contact with Dr. Chubin continued 
after he took a position with AAAS (see below). NACME representatives participated in two of 
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CAEE’s Advisory Board meetings: James Jones, Senior Director, in 2005, and Melonia 
Guthrie, Program Manager for University Programs, in 2007. CAEE leadership also discussed 
aspects of our research into diversity in engineering education in a series of phone calls with 
other senior NACME personnel, including VPs Tom Davis and Aileen Walter. 

American Association for the Advancement of Science (AAAS) 

Daryl Chubin, Director of the AAAS Center for Advancing Science & Engineering Capacity, 
attended several team meetings and the 2006 Advisory Board meeting. He also provided 
several reports for CAEE, including a “big picture" look at CAEE research directions as well as 
a guide to diversity-related resources that would be helpful in our research. 


CAEE had two significant relationships with companies who were very supportive of our 
research into undergraduate engineering education. 

Hewiett-Packard 

Hewlett-Packard became involved in 2003 through their University Relations group, sending 
engineering representative William Wickes to APS data workshops. Bill also reviewed an 
early draft of the PIE survey from the perspective of a practicing engineer. 

The Boeing Company 

John Craig, the Boeing engineering focal contact for the University of Washington’s Center for 
Engineering Learning and Teaching, participated in several APS data workshops held in the 
Seattle area, beginning in 2004. He also met with the Engineering Teaching Portfolio 
Program team to discuss possible extensions to the project, including a portfolio tool adapted 
for industry use. Throughout the period 2004-2010, John served as a sounding board for 
our results, drawing on his experience as a chief engineer in charge of a large group of 
engineers. He has provided valuable insights into CAEE’s work on the learning experiences 
of students and the transition of graduates to their first engineering Jobs. 


APPLE Survey Coilaborators 

Finally, the group of researchers at the 21 institutions who participated in the Broader 
National Sample (the APPLE Survey administered in spring 2008) is a significant example of 
CAEE’s collaboration efforts. This branch of the research was originally planned for only four 
other campuses but was expanded to 21 in early design modifications. These collaborating 
individuals were the primary contacts on each campus, responsible for leading the local 
activities, including the critical tasks of recruiting student participants and helping arrange 
details of the survey administration. The APS team produced individual reports for each of 
the 21 campuses, and these served as the basis for discussions about the results among the 
campus survey leads and their engineering administrators. In one instance, a CAEE co-PI 
was invited to present a workshop using the broader APS results on the participating APPLES 
campus, and this further collaboration led to a Joint paper at ASEE (Waters, Chen, and 
Sheppard 2010). 
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7 Looking Ahead: Ideas for Further Work 


In this section we discuss ideas for further work on being and becoming an engineer, an 
engineering educator, and an engineering education researcher. 

This report described four different components of CAEE’s work. It presented findings from 
two research studies: the Academic Pathways Study (APS) and the Study of Engineering 
Educator Decisions (SEED). It also described two programs: the Engineering Teaching 
Portfolio Program (ETPP) and the Institute for Scholarship in Engineering Education (ISEE). 
The following subsections discuss ideas for looking ahead to facilitate change on engineering 
campuses and for conducting future research in engineering education. We close by briefly 
discussing relevant research infrastructure issues and recalling the larger picture within 
which research on engineering education takes place. 


7.1 Effecting Change on Engineering Campuses 

Our hope is that the results from our work will inspire conversations and ensuing actions that 
will lead to change on engineering campuses. 

To facilitate those conversations, in addition to presenting the findings in this report, we offer 
the following: 

• Research briefs that present a short overview of many of the research papers that can 
be viewed on the CAEE web site at http://www.engr.washington.edu/caee/ 
portal_research_briefs.html. 

• A set of research-based questions that can be used to engage in conversations about 
student learning on engineering campuses. These questions are motivated by the 
summary of the APS results in Subsection 2.10 and presented as a full set of 
questions in Appendix D. 

• Materials to run workshops that can be used to present specific research findings to 
interested faculty and administrators. These workshops have been successfully run 
with engineering educators and faculty development professionals. Workshops are 
listed in Appendix A, Subsection A.3. 

• A set of research instruments that campuses interested in collecting their own data 
can use. The instruments are described in this report in Section 6 and on the CAEE 
web site at http://www.engr.washington.edu/caee/APS_Process_Procedures.html. 

• Materials from the Engineering Teaching Portfolio Program on our web site at 
http://www.engr.washington.edu/caee/etpp-sessions.htm for campuses seeking to 
prepare graduate students for teaching in a cost-effective way. 
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12 . Ideas for Future Research 

The Academic Pathways Study guides us to certain areas of educational research that 
warrant further analysis. For example, considering the diverse programs and student 
perspectives we observed across the institutions we studied, it makes sense that other 
institutions will have their own nuances to be explored. We also observed substantial 
changes in students between their first and senior years, both in terms of learning and 
development. This warrants further inquiry into the middle years— the experiences of 
sophomore and junior level students. Further, APS longitudinal research focused on the 
experiences of students who spend the entirety of their undergraduate careers at one 
institution. However, APS cross-sectional research shows that this represents only a portion 
of engineering students. Studies of community college and transfer students are becoming 
important as students increasingly follow this academic path. Finally, there are important 
questions concerning students who never consider or enter engineering. 

Given that the current Studies of Engineering Educator Decisions work focused on 
participants from a single institution, extending the research to confirm or refine findings 
reported here would be valuable. Such additional data could also be used to further 
investigate issues such as the role of research results in informing teaching decisions; the 
relationship between satisfaction, dissatisfaction, and additional change; and the broad 
issue of how engineering educators conceptualize students. 

Building on the Engineering Teaching Portfolio Program efforts, we could further explore the 
ways in which the portfolio construction activities help participants reflect on their existing 
ideas about teaching and ultimately develop a more sophisticated, integrated, personalized, 
and actionable understanding of teaching. Building on research on the community of 
engineering education scholars that was conducted as part of the Institutes for Scholarship 
in Engineering Education, we could expand our investigations into how people enter and 
navigate the field of engineering research, even as the community itself is growing. 

Appendix E represents an effort to move a conversation about future work beyond a direct 
extension of our work to a more expansive conversation about new research. In particular, 
the appendix contains research questions in the following seven areas: 

1. On pathways: These questions follow directly from the theme of the Academic 
“Pathways” Study. The questions relate to students moving into, through, and out 
of engineering education systems. 

2. On learning engineering: These questions focus on multiple aspects of the 
engineering education experience. 

3. On significant learning experiences: The questions in this category draw attention 
to specific types of experiences that students have and the benefits of these 
experiences. 

4. On engineering knowing: The questions in this category raise issues related to the 
underlying nature of engineering knowing, and how answers concerning what 
counts as engineering knowing can influence other curricular decisions. 

5. On teaching engineering students: This category contains questions related to how 
engineering educators engage in teaching activities and how to help them engage 
more effectively. 

6. On researching issues in engineering education: This category includes questions 
on how to support engineering education researchers and how the process of 
engaging in such research might affect the researcher. 
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7. On bringing about change in engineering education: The final set of questions 
raises issues concerning the phenomenon of change in academic settings. 

The research questions in each category were assembled with four design principles in mind. 
First, in each category, we start with questions that are directly grounded in CAFE research 
results. In some of the categories, later questions introduce broader issues that have arisen 
during the course of our research on the student learning experience, the faculty teaching 
experience, and the community of scholars in engineering education. Second, we 
intentionally produced sets of questions that are extensive and far reaching. This is in 
keeping with our goal of stimulating new ideas and debate in the growing, interdisciplinary 
community that is engaged in scholarship in engineering education. Third, we include 
questions that are at multiple levels of detail, in order to satisfy the interests of a wide range 
of readers. Finally, each category includes many types of questions. Some of the questions 
seek understanding of engineering students, engineering educators, and engineering 
researchers on dimensions examined in our existing work. Others concern explanations for 
specific CAFE findings, possible interventions, and more overarching philosophical issues. 


7.3 A Research Infrastructure for Scholarship in 
Engineering Education 

To engage in scholarship to address the range of issues in the list of future research 
questions presented in Appendix E, there is need for a sound research infrastructure. 

Central to this infrastructure is a vibrant and engaged community of individuals with an 
appropriate range of expertise to engage in the work. This infrastructure also includes tools 
(e.g., communication, computing, databases) to enable the increasingly interdisciplinary and 
geographically distributed set of individuals in the community to collaborate to conduct 
scholarship in engineering education. 

The community of individuals has been expanding in recent years. The CAEE team began 
work in January 2003. In 2010, at the end of the seven years that the Center was funded, 
the engineering education community is much larger, more distributed, more 
interdisciplinary, and it has expertise in a larger set of research methods. Building on the 
dedicated set of individuals in the Educational Research and Methods division of the 
American Society for Engineering Education (ASEE), there has been rapid growth in the field 
in the last decade (Adams and Cummings-Bond 2004; Adams et al. 2006; Atman 2006; 
Atman, Engineering education research, 2007; Streveler and Smith 2006; Sheppard et al. 
2008; Jamieson and Lohmann 2009). 

When initiated in 2003, CAEE joined the ranks of several centers with a focus on engineering 
education, including the NSF-funded Vanderbilt-Northwestern-Texas-Harvard/MIT 
Engineering Research Center (VaNTH, http://www.vanth.org/) and the NAE’s Center for the 
Advancement of Schoiarship in Engineering Education (CASEE, http://www.nae.edu/casee), 
as weii as a growing number of campus based centers such as the Center for Engineering 
Learning and Teaching at the University of Washington (http://depts.washington.edu/ 
ceitweb/), the Leonhard Center at Pennsylvania State University (http://www.engr.psu.edu/ 
leonhardcenter/), and the Center for Research on Learning and Teaching North at the 
University of Michigan (http://www.engin.umich.edu/teaching/crltnorth/). In addition to 
growth in the number of centers, engineering education departments have been estabiished 
on such campuses as Purdue, Virginia Tech, Utah State, and Clemson. Growth has not been 
iimited to colieges of engineering. There are now facuity positions with a focus on 
engineering education at several colleges of education across the country. 
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CAEE contributed to this growing field not only through the generation of research results, but 
also by directly contributing to the growth of the community through ISEE and through the 
large number of individuals who were involved directly with the Center. 

While the number of issues that are pressing for research in engineering education grows, so 
does the community of scholars who are not only interested but also have the expertise to 
address those problems. 


7.4 Remembering the Big Picture 

Engineering education is a rich and vibrant area for research, with many opportunities for in- 
depth scholarship that can contribute significantly to improving engineering education for 
many constituencies. It is important to remember that the research is intended to support a 
variety of actors in the engineering education system to work towards goals that represent 
improvement in the system. 

The breadth of potential contributions and constituencies can be seen in the ways that one 
could fill in the blanks in the statement, “Through our research, we are creating 1 for 
2 to enable 3 Figure 7.4-A lists some of the ways that such a prompt can be 
populated. Combining the three elements in various ways demonstrates the breadth of the 
pathways for change. To illustrate, combinations include (a) individual students using stories 
of other students to make informed decisions about their own pathways, (b) classroom 
educators using findings about student motivation to increase the effectiveness and 
inclusiveness of their teaching, and (c) policy makers using information about supports and 


Figure 7.4-A: Filling in the blanks in the statement, “Through our research, we are 
creating 1 for 2 to enable 3 (Smith et at. 2004) 


1: foundation, database, pictures, stories, opportunities, expectations, revelations, problems, 
questions, relationships, lessons learned, pathways, resources, ideas, accounts, descriptors, 
processes, definitions, scenarios, grounded theory, research models, empirical claims, tools, 
instruments, quick read, publication series on engineering education, quality control model for 
engineering education, confidence, legacy, learning career guide for students, misconceptions, 
future research directions, roadmap, territory, "how people learn engineering,” forum, venue, 
place, context 

2: U.S., students, engineers, teachers, policy makers. National Science Foundation, 
educational programs, education institutions, industry, learning sciences community, 
engineering faculty, faculty, potential students, guidance counselors, teachers in the U.S., U.S. 
president, employers. National Research Council, National Academy of Engineering, CAEE 
team, American Society of Engineering Education, European community, engineering 
professional societies, educational technology, parents, globe, mentors (coaches), community, 
administrators, legislature, the electorate, educational foundations, higher education 
community, corporate presidents, community colleges, student organizations 

3: better learning, change, better society, informed electorate, broadly thinking engineers, new 
research directions, better teaching, better mentoring, better retention, better recruiting, 
clearer pathways, better self-understanding/awareness, knowledge base, awareness of 
diversity in many senses, multiple pathways, autonomous learning, enable learning how to be a 
good student, better policy, diversity in pedagogical practices, success for all, concept based 
learning, better educational support, improved self-efficacy/confidence, better engineering 
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barriers to structure the entire educationai experience. 

CAEE contributed to the field of engineering education during our seven years of funding, not 
only through the generation of a rich body of research results, but also by demonstrating the 
scope of research and program activity that a large center is uniquely capable of 
accomplishing. CAEE also contributed to the growth of the engineering education community 
through the many individuals who were involved directly and collaboratively with the Center 
and through the community-building efforts of ISEE. 

Moving forward, the engineering education community needs to consider a large range of 
challenges, issues, stakeholders, constituencies, and opportunities involved with and 
affected by improving engineering education. In closing, we anticipate further research- 
based improvements to engineering education, ensuring that a diverse cadre of engineering 
graduates are prepared for the challenges they will face in the coming years. 
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8 Dedication 


This report is dedicated to the memory of Denice D. Denton. 
Denice was with us through the initial years of CAEE. During 
this period, she was a team member in every sense of the word. 
She stayed up late nights with us as we wrote our proposal and 
conducted our reverse site visit at the National Science 
Foundation. She helped us to organize our research and was 
continually an inspiration as she reminded us about the larger 
vision of why we were doing this work: to improve the 
engineering education environment for all students. 

We hope that our work will honor Denice’s memory and help 
carry on her commitment to creating a diverse cadre of 
engineering students who are ready to meet the challenges of 
the 21st century. 




The CAEE team (above) and Denice Denton (above right) at the 2003 spring 
kick-off meeting 
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B Appendix: 

Cumulative Team List and 
Advisory Board Members 


The CAEEteam consisted of faculty, postdoctoral research associates, graduate research 
assistants, undergraduates, and professional staff at the five CAEE partner campuses, as 
well as at many additional campuses. In addition. External and Internal Advisory Boards 
provided guidance and oversight of Center activity. 
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academic title and Institution for the date of their last affiliation with the Center. Graduate 
research assistants are listed separately, but undergraduate students are not listed by name. 
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Robin Adams, Ph.D.* 

Purdue University 

Assistant Professor, Engineering 

Education 

ISEE Lead, APS 

2003 - 2010 

Jeff Aldrich 
Stanford University 
IT Consultant 
2003 

Cheryl Allendoerfer, Ph.D. 
University of Washington 
Research Scientist 
ISEE 

2004 - 2008 


Ravel Ammerman, Ph.D. 

Colorado School of Mines 
Lecturer, Division of Engineering 
APS 

2004-2006 

Daniel Amos, Ph.D. 

University of Washington 
Research Scientist 
APS 

2006-2008 

Cindy Atman, Ph.D. 

University of Washington 

Professor, Human-Centered Design and 

Engineering 

CAEE PI, Director, Campus PI 
APS, SEED, ISEE 
2003-2010 
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Sylvia Bach 

University of Washington 
CAEE Administrator 
2003-2010 

Caryn Bailey, Ph.D. 

Howard University 
Research Scientist 
APS 
2003 

Shelley Balanko, Ph.D. 

University of Washington 

Program Evaluator, Office of Educational 

Assessment 

2003 

Philip Bell, Ph.D. 

University of Washington 
Associate Professor, Educational 
Psychology 
ISEE Principal, APS 

2003 - 2005 

Karen Bland, Ph.D. 

Howard University 
Research Scientist 
APS 

2006-2007 

Jim Borgford-Parnell, Ph.D. 

University of Washington 

Assistant Director, Center for Engineering 

Learning 

and Teaching (CELT) 

APS 

2005 - 2010 

Wade Boykin, Ph.D. 

Howard University 
Professor, Psychology 
APS 
2003 

Kimberley Breaux, Ph.D. 

Colorado School of Mines 
Research Scientist 
APS 

2004 - 2007 

Debbie Chachra, Ph.D. 

Franklin W. Olin College of Engineering 
Assistant Professor of Materials Science 
APS 

2006 - 2010 


Helen Chen, Ph.D. 

Stanford University 

Social Science Research Associate 

APS 

2003-2010 
Mia Clark 

Stanford University 
Technical Writer 
APS 

2006-2008 

Angela Cole, Ph.D. 

Howard University 

Assistant Professor, Psychology 

APS 

2003-2004 

Laurie Collins, Ph.D. 

University of Washington 

Program Evaluator, Office of Educational 

Assessment 

2003 

Gloria Dang 

University of Washington 
Office Assistant 
2003-2006 

Krista Donaldson, Ph.D. 

Stanford University 
Research Scientist 
APS 

2006-2008 

Kimarie Engerman, Ph.D.* 

University of the Virgin Islands 
Assistant Professor, Psychology 
APS, ISEE 
2003-2006 

Ozgur Eri§, Ph.D.* 

Franklin W. Olin College of Engineering 
Assistant Professor, Mechanical 
Engineering and Design 
APS 

2003-2010 

Lorraine Fleming, Ph.D. 

Howard University 
Professor, Civil Engineering 
Co-Pi, Campus PI 

APS principal, ISEE Principal, ETPP 
2003-2010 
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Shannon Gilmartin, Ph.D. 

Stanford University 
Consulting Assistant Professor, 
Mechanical Engineering 
APS 

2008-2010 

Jerry Gilmore, Ph.D. 

University of Washington 
Evaluation Consultant 
2007-2010 

Patricia Gomez 
University of Washington 
Assistant to the Director 

2005-2009 

Tawni Hoeglund, Ph.D. 

Colorado School of Mines 
Research Associate 
APS 

2004 - 2006 

Yi-Min Huang-Cottrille, Ph.D. 

University of Washington 
Research Scientist 
ETPP, SEED 

2005 - 2008 

Andy Jocuns, Ph.D. 

University of Washington 
Research Associate 
APS 

2007-2010 

Deborah Kilgore, Ph.D. 

University of Washington 
Research Scientist 
APS 

2005-2010 

Russ Korte, Ph.D.* 

University of Illinois 

Assistant Professor, Human Resources 

Development 

APS 

2005-2010 

Sislena Ledbetter, Ph.D. 

Howard University 
Research Associate 
APS 

2007-2008 


Larry Leifer, Ph.D. 

Stanford University 

Professor, Mechanical Engineering Design 
Co-PI 

ISEE Principal, APS 
2003-2008 

Gary Lichtenstein, Ed.D. 

Stanford University 
Consulting Associate Professor 
APS 

2005 - 2010 

Jennifer Light, Ph.D.* 

Lewis-Clark State College 

Research Associate (NAE CASEE Fellow) 

APS 

2006 - 2007 

Angela Linse, Ph.D. 

University of Washington 
ETPP co-Lead, APS 
2003-2004 

Heidi Loshbaugh, Ph.D. 

Colorado School of Mines 
Research Scientist 
APS 

2003-2007 

Tina Loucks-Jaret 

University of Washington 

Technical Communications Specialist 

2006 - 2009 

Dennis Lund 
University of Washington 
Assistant Director 

2003 - 2010 

Bayta Maring, Ph.D. 

University of Washington 

Program Evaluator, Office of Educational 

Assessment 

2004 - 2010 

Holly Matusovich, Ph.D.* 

Virginia Tech 

Assistant Professor, Department of 

Engineering Education 

APS 

2007 - 2010 
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Janice McCain, Ph.D. 

Howard University 
Research Associate 
APS, ISEE 

2006 - 2009 

Susan Millar, Ph.D. 

University of Wisconsin 
Senior Research Advisor 
External Evaluator 

2003-2007 

Ronald Miller, Ph.D. 

Colorado School of Mines 
Professor, Chemical Engineering 
Campus PI, APS 

2003 - 2010 

Elizabeth Moore, Ph.D. 

University of Washington 

Program Evaluator, Office of Educational 

Assessment 

2004 - 2010 

Quan Nguyen 
University of Washington 
Office Assistant 

2007 - 2010 

Kevin O’Connor, Ph.D.* 

University of Rochester 

Assistant Professor, Warner Graduate 

School of 

Education and Human Development 
APS 

2003-2007 

Barbara Olds, Ph.D. 

Colorado School of Mines 
Professor Emerita, Division of Liberal Arts 
and International Studies 
APS, SEED 

2003 - 2010 

Julie Provenson 
University of Washington 
Assistant to the Director 
2009-2010 

Natalie Quilter 
University of Washington 
Assistant to the Director 

2004 - 2005 


Portia Sabin, Ph.D. 

University of Washington 
Research Associate 
APS 

2005 - 2006 

Jason Saleem, Ph.D. 

University of Washington 
Research Scientist 
APS 

2006 - 2007 

Sheri Sheppard, Ph.D. 

Stanford University 

Professor, Mechanical Engineering Design 

CAEE Co-PI, Campus PI 

APS Lead, SEED Advisor, ISEE Advisor 

2003-2010 

Karl Smith, Ph.D. 

Purdue University 
Cooperative Learning Professor, 
Engineering Education 
CAEE CO-PI, Campus PI 
APS, SEED, ISEE Advisor 
2003-2010 

Reed Stevens, Ph.D.* 

Northwestern University 

Professor, Education and Social Policy 

CAEE Co-PI 

APS Principal 

2003-2010 

Ruth Streveler, Ph.D. 

Purdue University 

Assistant Professor, Engineering 

Education 

CAEE Co-PI 

APS Principal 

2003 - 2010 

Candace Sulzbach, Ph.D. 

Colorado School of Mines 
Lecturer, Division of Engineering 
APS 

2004 - 2006 

George Toye, Ph.D. 

Stanford Center for Design Research 

Database Consultant 

APS 

2003-2010 
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Jennifer Turns, Ph.D. 

University of Washington 

Associate Professor, Human Centered 

Design & Engineering 

SEED and ETPP Lead, APS Advisor, ISEE 

Advisor 

2003-2010 

Dawn Williams, Ph.D. 

Howard University 
Associate Professor, Educational 
Administration and Policy 
APS 

2006-2010 


Ken Yasuhara, Ph.D.** 
University of Washington 
Research Scientist 
APS 

2003 - 2010 

Jessica Yellin, Ph.D. 
University of Washington 
Research Scientist 
ETPP, SEED 

2004 - 2008 


B.2 Graduate Research Assistants 


Tori Bailey 
Stanford University 
APS 

2003-2007 

Theresa Barker 
University of Washington 
APS 

2003-2005 

Samantha Brunhaver 
Stanford University 
APS 

2009-2010 

Tameka Clarke Douglas 
Colorado School of Mines 
Difficult Concepts Study 
2008-2010 

Laura Crenwelge 
Stanford University 
APS 

2006 - 2007 

Rebecca Currano 
Stanford University 
ISEE 

2005-2007 

Katherine Deibel 
University of Washington 
APS 

2007 - 2010 


Matt Eliot 

University of Washington 
ETPP 

2004 - 2006 

John Feland 
Stanford University 
APS 
2003 

Lari Garrison 
University of Washington 
APS 

2003-2010 
Monica Geist 

University of Northern Colorado 
APS 

2005 - 2009 

Ashley Griffin 
Howard University 
APS 

2003-2005 

Kathleen Gygi 
University of Washington 
SEED 

2009-2010 

Kristyn Jackson 
Stanford University 
APS 

2007-2008 
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Jana Jones 

University of Washington 
ETPP 

2003-2004 

Phyllis Randle 
Howard University 
APS 

2007-2008 

Marcus Jones 
Howard University 
APS 

2005-2007 

Rashika Rentie 
Howard University 
APS 

2006-2008 

Russ Korte** 

University of Minnesota 
APS 

2005-2008 

Ed Rhone 

University of Washington 
APS 

2003-2004 

Micah Lande 
Stanford University 
APS 

2005-2010 

Emma Rose 

University of Washington 
SEED 

2005-2006 

Steve Lappenbusch 
University of Washington 
ETPP 

2006-2007 

Aidsa Santiago Roman 
Colorado School of Mines 
Difficult Concepts Study 
2008-2010 

Holly Matusovich** 
Purdue University 
APS 

2007-2010 

Brook Sattler 
University of Washington 
SEED 

2006-2010 

David Mitchell 
Howard University 
APS 

2006-2007 

Tom Satwicz 
University of Washington 
APS 

2003-2004 

Andrew Morozov 
University of Washington 
APS 

2007-2009 

Katie L. Schatz 
University of Washington 
ISEE 

2003-2004 

Elizabeth Otto 
Stanford University 
APS 

2009-2010 

Derek Seward 
University of Rochester 
APS 

2006-2007 

Sarah Parikh 
Stanford University 
APS 

2008-2010 

Carmen Smith 
University of Washington 
APS 

2003-2004 

Lisa Perhamus 
University of Rochester 
APS 

2006-2007 

Andrene Taylor 
Howard University 
APS 

2006-2007 
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Tiffany Tseng 
Stanford University 
APS 

2009-2010 

Katherine Winters 
Virginia Tech 
APS 

2009-2010 

Heather Toomey-Zimmerman 
University of Washington 
APS 

2004-2006 

Tammy VanDeGrift 
University of Washington 
ETPP 

2003-2005 

Ken Yasuhara** 
University of Washington 
APS 

2003-2010 

B.3 ISEE Institute Scholars 



Faculty and graduate students who participated in the Institute for Scholarship on 
Engineering Education are listed below by cycle year with their institutional affiliation at the 
time of ISEE participation. See Section 5 for details about ISEE. 


2004-2005 Institute Scholars 

Joe Cannon 
Howard University 

Lawrence Neeley 
Stanford University 

Russ Caspe 

University of Washington 

Tori Rhoulac Smith 
Howard University 

Eric Cheek 

North Carolina A&T State University 
Scott Eberhardt 

Louis Rosenberg 

California Polytechnic State University, 
San Luis Obispo 

University of Washington 
Brian Fabien 

David Socha 
University of Washington 

University of Washington 
Brian Flinn 

Jeremy Sabol 
Stanford University 

University of Washington 
PK Imbrie 

Tom Williams 
University of Washington 

Purdue University 
Linda Lee 

Denise Wilson 
University of Washington 

University of Washington 

Maisy McGaughey 
University of Washington 

Ken Yasuhara 
University of Washington 

2005-2006 Institute Scholars 

Frank Ashby 

University of Washington 

Rebecca Currano 
Stanford University 

Tawana Carr 
Howard University 

Greg Deierlein 
Stanford University 
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Robyn Dunbar 
Stanford University 

Lisa Hansen 

University of Washington 

Marcus Jones 
Howard University 

Cathryne Jordan 
University of Washington 

Russ Korte 

University of Minnesota 

Micah Lande 
Stanford University 

2006-2007 Institute Scholars 

Caroline Baillie 
Queen’s University 

Rebecca Bates 

Minnesota State University, Mankato 

Angela Bielefeldt 
University of Colorado, Boulder 

Larry Bland 

John Brown University 

Kami Carey 
Howard University 

Marcel Castro 
Howard University 

George Catalano 

Binghamton University, State University of 
New York 

Jaime Hernandez Mijangos 
Texas State University, San Marcos 

Karen High 

Oklahoma State University 


Reginald Mitchell 
Stanford University 

David Prince 
University of Washington 

Jeremy Sabol 
Stanford University 

Ross Shachter 
Stanford University 

Karl Smith 

University of Minnesota 

Simon Wong 
Stanford University 


Sharon Jones 
Lafayette College 

Stephanie Luster-Teasley 
North Carolina A&T State University 

Kristyn Masters 

University of Wisconsin, Madison 

James McGuffin-Cawley 
Case Western Reserve University 

Lorelle Meadows 

University of Michigan, Ann Arbor 

Donna Michalek 

Michigan Technological University 
Jill Nelson 

George Mason University 
Sean St. Clair 

Oregon Institute of Technology 

Carol Stwalley 
Purdue University 
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B.4 External Advisory Board Members 

The CAEE External Advisory Board consisted of engineering deans, educators, and industry 
representatives from around the U.S. who provided guidance and oversight of CAEE activity. 
Board members met annually with the CAEE team from 2003 to 2007. (The two-year 
extension for added work on APS did not include provisions for an advisory board.) The 16 
people listed below served on the External Advisory Board during 2003-2007, with 9 to 13 
active members at any one time. 


David Wormley, Ph.D., Chair 
The Pennsylvania State University 
Dean of Engineering 
(2003-2007) 

Susan Ambrose, Ph.D. 

Carnegie Mellon University 
Associate Provost for Educational 
Development 

Director, Eberly Center for Teaching 
Excellence 

Principal Lecturer, Department of History 
(2003-2007) 

Denice Denton, Ph.D. 

University of Santa Cruz 

Chancellor 

(2005-2006) 

Richard Felder, Ph.D. 

North Carolina State University 
Professor Emeritus of Chemical 
Engineering 
(2003-2007) 

Thomas Foot 

Ford Motor Company, retired 
Manager, Super Duty Powertrain 
(2003-2006) 

Norman Fortenberry, Ph.D. 

National Academy of Engineering 
Director, Center for the Advancement of 
Scholarship on Engineering Education 
(2006-2007) 

Louis Gomez, Ph.D. 

Northwestern University 

Professor, Education and Social Policy 

(2006-2007) 

Mario Gonzalez, Ph.D. 

University of Texas-Austin 
Professor, Electrical & Computer 
Engineering, retired 
(2004-2007) 


Rogers Hall, Ph.D. 

Vanderbilt University 

Professor, Department of Teaching and 

Learning 

(2003-2007) 

Carolyn Meyers, Ph.D. 

North Carolina A&T State University 
Provost and Vice Chancellor for Academic 
Affairs 

(2003-2006) 

Gregory Moses, Ph.D. 

University of Wisconsin 
Professor, Engineering Physics 
(2003-2007) 

Alfred Moye, Ph.D. 

Hewlett-Packard Company 
Director of University Affairs, retired 
(2003-2007) 

John W. Prados, Ph.D. 

University of Tennessee 
Vice President Emeritus 
Professor, Chemical Engineering 
(2003-2007) 

John Roundhill 

The Boeing Company 

Vice President for Product Strategy & 

Development, retired 

(2003-2007) 

Elaine Seymour, Ph.D. 

Director Emeritus and Research Associate: 
Ethnography & Evaluation Research 
Center to Advance Research and Teaching 
in the Social Sciences (CARTSS) 
(2003-2006) 

Karan Watson, Ph.D. 

Texas A&M University 
Associate Provost and Dean of Faculties 
Professor, Electrical Engineering 
(2003-2007) 
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B.5 Internal Advisory Board Members 


The CAEE Internal Advisory Board provided advice and support for CAEE activity on the five 
partner campuses. The Internal Advisory Board heid periodic phone conferences and was 
chaired by an administrator from the University of Washington’s Coiiege of Engineering. 


Matt O’Donneil, Ph.D., Chair 
University of Washington 
Dean, Coiiege of Engineering 
( 2007 ) 

John Bransford, Ph.D. 

University of Washington 
Professor, College of Education 
( 2004 - 2007 ) 

Steven L Crouch, Ph.D. 

University of Minnesota 
Dean, Institute of Technology 
( 2005 - 2007 ) 

H. Ted Davis, Ph.D. 

University of Minnesota 
Dean, Institute of Technology 
( 2003 - 2004 ) 

Denice D. Denton, Ph.D. 

University of Washington 
Dean, Coiiege of Engineering 
( 2003 - 2004 ) 

Peter Hudieston, Ph.D. 

University of Minnesota 
Associate Dean for Student Affairs, 
Institute of Technoiogy 
( 2004 - 2007 ) 

James H. Johnson, Jr. Ph.D. 

Howard University 

Dean, Coiiege of Engineering, Architecture 

& Computer Sciences 

( 2003 - 2007 ) 


Nigei Middleton, Ph.D. 

Colorado School of Mines 

Vice President for Academic Affairs 

( 2003 - 2007 ) 

Jody D. Nyquist, Ph.D. 

University of Washington 
Associate Dean, Graduate Schooi 
( 2003 - 2007 ) 

James D. Plummer, Ph.D. 

Stanford University 

Dean, School of Engineering 

( 2003 - 2007 ) 

Mani Soma, Ph.D. 

University of Washington 

Acting Dean, Coiiege of Engineering 

( 2005 - 2006 ) 

Orlando Taylor, Ph.D. 

Howard University 
Dean, Graduate Schooi 
( 2003 - 2007 ) 

Patricia A. Wasley, Ph.D. 

University of Washington 
Dean, Coiiege of Education 
( 2003 - 2007 ) 

Donaid H. Wulff, Ph.D. 

University of Washington 
Associate Dean, Graduate School 
Director, Center for Instructionai 
Development and Research 
( 2003 - 2007 ) 
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C Appendix: APS Headlines 


This appendix provides an at-a-glance summary of Academic Pathways Study findings 
presented in Section 2 (Student Learning Experiences) in the form of a compilation of the 
“headlines” or lowest-level subsection headings. Many headlines summarize claims, while 
others identify specific topics addressed by the APS findings. The introductory and 
concluding subsections (2.1, Overview of the APS, and 2.10, Enabling Success for 
Engineering Students) were excluded from this compilation. 


2.2 The College Experience 

Engineering majors are as likely to persist as are other majors. 

There are similarities among, but also differences between, engineering majors and other 
majors with respect to learning and college-experience measures. 

Engineering persisters are more likely to be male and white, and less likely to be first- 
generation college students. 

Women are more likely to migrate into engineering. 

Where do the switchers go? Where do engineering in-migrators come from? 

On many measures, engineering persisters and switchers are similar. 

On some measures, persisters and switchers are different. 

Persisters and switchers differ in intention to complete an engineering major. 
Commitment of persisters increases over the four years. 

Entering students interested in engineering often have limited knowledge of engineering. 
Range of intentions to complete an engineering major 

Level of commitment to engineering depends on students’ identification with engineering 
activities. 

2.3 Motivation to Study Engineering 

Top motivational factors are behavioral, psychological, social good, and financial. 

Mentors and parents are less salient motivators. 

Motivational factors are interrelated. 

Motivation remains essentially constant over the four undergraduate years. 

Other aspects of motivation: status, portability, and “sticking it out" 

Motivation varies with gender and major. 

Motivation is correlated with persistence and satisfaction. 

Identity development as an engineer viewed within a framework of sponsorship 
An example of a lack of engineering sponsorship for a student’s interests 
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2.4 The Engineering College Experience 

Positive differences between seniors and first-years 
Negative differences between seniors and first-years 
Women and men, alike. ..and different 

Identification with engineering: Variations in perceptions of personal cost, enjoyment, and 
future usefulness 

Transfer students’ experiences 

Socioeconomic status 

On some measures, the groups are the same. 

Synthesizing the differences to characterize each group 

High psychologicai motivation, high professional/interpersonal confidence (M/C) 

Low psychologicai motivation, iow professional/interpersonal confidence (m/c) 

High psychologicai motivation, low professional/interpersonal confidence (M/c) 

Low psychological motivation, high professional/interpersonal confidence (m/C) 
Demographics by group 
An emerging picture of involvement 

2.5 Engineering Knowledge, Conceptions, and Confidence 

Students’ understanding of engineering disciplinary knowledge changes over time. 

Some students struggle with the shift from “book problems” to open-ended problems. 

Use of engineering-specific language increases during the undergraduate years. 

Students’ knowledge of engineering does grow from first to senior year. 

Co-ops and internships buiid knowledge of engineering. 

Many seniors did not perceive gaining knowledge of engineering from schooi-related 
experiences. 

Are capstone projects not realistic and too iate in the curriculum? 

Students recognize different skills as important in engineering. 

Learning about engineering is mostly similar among women and men. 

Importance of and preparedness with engineering skilis and knowledge 
Not all confidence ievels are equai; women’s confidence iags in some areas. 

Students exhibit iow confidence in professionai and interpersonal skills. 

Confidence in math and science skills remains constant. 

Non-engineering factors iargeiy contribute to confidence in interpersonai skilis. 

Possible explanations for differences in perceived importance and confidence in key skills 
Even graduating seniors misunderstand some key engineering concepts. 

Faculty are often unaware of misunderstandings and the difficulty of these concepts. 
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2.6 Engineering Design Knowiedge, Conceptions, and Confidence 
Conceptions of engineering design shift during the undergraduate years. 

Conceptions of engineering design vary with gender and institution. 

Men report higher confidence and course preparation with design than women. 
First-year students consider design problem context. 

Many sophomores do not consider design problems In temporal context. 

Students do not consider broad context more as juniors and seniors than as first-years. 
Consideration of temporal context does not develop significantly by senior year. 

Women are more likely to consider certain aspects of broad context during design. 

2.7 Looking Beyond Graduation: Student Pians 

Nearly 80% said “yes” to engineering work, and 20% were unsure or leaning away. 
Co-ops, internships influence post-graduation plans to pursue engineering. 

Forty percent considering engineering graduate school 
Seniors still unsure about their plans 

More than 60% of engineering graduates had a combination of plans. 

An engineering degree can provide a basis for many future options. 

URM students were initially more interested in engineering graduate school. 

URM women and men think differently about post-graduation options. 

Women’s plans similar to men’s, but... 

Psychological motivation/interest an important factor 

Confidence in professional and Interpersonal skills an important factor 

Institutional differences can have strong influences on student pathways. 

2.8 Looking Beyond Graduation: Experiences in the Work Worid 

Technical problems are more complex and ambiguous in the work world. 

Many different players and processes can affect decisions. 

Support from managers and coworkers is very important and can vary greatly. 
Differences in age or outside interests can impede camaraderie. 

Rotation of new engineers can inhibit forming strong relationships with coworkers. 
Teamwork was much different in the workplace than In school. 

Understanding one’s role 

Getting a sense of the bigger picture 

Company education efforts could be insufficient 

The importance of communication and documentation 

Communicating with non-engineers 
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Learning to use a new language 

2.9 Summarizing Resuits about Diversity 

Gender, motivation, and approaches to engineering 

Gender, motivation, and major 

Gender, URM, and mentor influence 

Gender and extracurricular activities 

Gender and curricular overload 

Gender and race/ethnicity in the classroom 

Gender and professional and interpersonal skills 

Gender and engineering knowledge gain 

Confidence in math and science skills 

Gender and conceptions of design 

Gender and confidence in design abilities 

Gender and approaches to design 

Graduate school 

Engineering work 
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D Appendix: Local Inquiry Questions 


The concluding subsection of Section 2 (Student Learning Experiences) presented a series of 
questions based on Academic Pathways Study research. These “iocal inquiry questions” 
were organized by topic and designed to facilitate reflection and discussion on how each 
topic plays out on an individual campus and in an individual classroom. This appendix 
consists of a compilation of these iocai inquiry questions. See Subsection 2.10 for these 
questions placed in the context of relevant APS findings summaries. In the word cloud 
below, word size is proportional to their frequency in the local inquiry questions (exciuding 
the two highest-frequency words, which were “students" and “engineering,” and stop words). 



http://www.wordle.net/ 


D.l Welcoming Students into Engineering 

• Informed Decision Making: Does your college offer courses or programs (such as 
speaker series) that reveal to students the range of jobs and careers within the 
engineering fieid? How are students encouraged to integrate a variety of experiences 
into informed decision making on majoring in engineering? Do they have an accurate 
and sufficient understanding of the field of engineering and their place in it? How is 
re-examination of their decisions to stay in engineering supported through advising? 

• Migration in: Are there opportunities in the first years of college at your school (such 
as “introduction to engineering” seminars or courses) that aiiow students to explore 
engineering? How much migration in is happening at your institution? How might this 
pathway be expanded? Are there institutional barriers that discourage students from 
transferring into engineering? 
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• Pathways: What is the range of pathways that your students take through your 
curricula? Where do they find support? What organizations, faculty, student groups, 
and peers help students navigate through the institution? Does your institution 
support varied pathways through the undergraduate experience? 


D.2 Understanding and Connecting with Today's Learners 

• Listening: How do you get feedback from students about the effectiveness of various 
elements of your program? Do faculty listen to students about the effectiveness of 
their teaching? What mechanisms can be put in place to encourage more timely and 
effective use of teaching evaluations by instructors? How can what is learned through 
evaluations be better aligned with program improvement? Do you provide an 
environment where students listen to each other? 

• Student Passion: What motivates students on your campus to choose an engineering 
program? What can they be passionate enough about to keep them in an engineering 
program? Does your program include elements that will ignite and sustain student 
passion? 

• Variability/Commonality: How are students in your college of engineering similar to 
one another? How are they different from one another? How well do faculty and 
policy makers on your campus understand similarity and variability in your students’ 
motivation, background, interests, learning challenges, confidence, and future plans? 

• Supporting Diversity: Do individuals from traditionally underrepresented populations 
feel supported and included in the engineering community on your campus? Do 
faculty, students, and administrators recognize and support the important voices 
brought to engineering from individuals of all backgrounds? 


D.3 Helping Students Become Engineers 

• Student Identity as an Engineer: Do the students you teach know what engineers 
really do? Do they identify themselves as engineers? How does your program help 
them do this? Can they articulate what they are bringing to the engineering 
profession? Do faculty and administrators think about a student’s engineering 
identity as an element of student development in the undergraduate years? 

• Connecting Across the Years: Does your college connect the early learning 
experiences in the first two years (math- and science-focused) to the more 
engineering-focused experiences in the later years? How do design experiences in 
upper-division courses build on design experiences in early courses? 

• Learning Engineering: How do you confirm that students have learned and retained 
the basic skills of engineering? Have your students acquired the language of 
engineering? Have they mastered the concepts that are difficult to understand? Can 
they define and solve engineering design problems? Do they have the skills and 
confidence to meet society’s grand challenges? 

• Well-Rounded: How broadly do engineering students on your campus conceptualize 
engineering? How many areas beyond math, science, and analysis would students list 
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as important components of engineering? How skiiled are your graduates in the many 
aspects of the engineering profession? 

• Designing in Context: Do your graduates have the design skills they need? Do your 
students consider the broad context of engineering problems as they solve them? Do 
they think about the users and other stakeholders of an engineered solution, and all 
aspects of the life cycle? Are they considering global, environmental, societal, 
economic, and cultural context in engineering design? 


D.4 Developing the Whole Learner 

• Balance: Are your students satisfied with their undergraduate experiences as 
engineering students? Are they abie to baiance between their engineering and non- 
engineering extracurricular activities? Is there balance between individual and team 
experiences, weli-defined and open-ended probiems, and design and analysis 
experiences? Are your students abie to find balance between the academic and 
sociai aspects of their lives? 

• Significant Learning Opportunities: How does your institution provide learning 
opportunities that students consider significant, including experiences that connect 
with what students find meaningful, present students with a challenge, ask students 
to be self-directed learners, give students ownership over their learning, and facilitate 
development of a broad vision of engineering? 

• In-Depth Learning Opportunities: Do your students have opportunities to have 
learning experiences that help them extend their understanding of engineering, e.g., 
internships, co-ops, research or international experiences, and project-based 
learning? Do you help your students reflect on these experiences and integrate them 
into their understanding of the engineering profession? How might these reflections 
be integrated into program assessment and improvement? 

• Learning Environment: How would you characterize the learning environment on your 
campus? Is there an atmosphere of students in competition with each other? Do 
students feel overloaded by a demanding curriculum? Do all students feel that your 
institution would like them to succeed? Do your students develop confidence in their 
abilities as engineers? Are your students excited when they graduate, or do they 
seem to be Just sticking it out to the end? 

• Asking Questions: Do your graduates recognize when they do not know something? 

Do they have the skills to find the answers to their questions? Do they feel enabled to 
continue the learning process after they graduate? 


D.5 Positioning Students for Professional Success 

• Post-Graduation Plans: What resources are available at the department, college, and 
institution ievels for guidance in Job and career planning? Do your students feei 
enabled to enter a variety of professions? Are they prepared to be effective in those 
professions? What pians do your graduating students have? Are they considering a 
career in engineering, another field, or both? Work in industry or the public sector? 
Graduate school in engineering or another field? 
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• Ability to Practice: What challenges do your graduates face when they begin practice 
or graduate school? What helps facilitate their transition? Do they know how to seek 
out the information and advice they need? Are they prepared for a career or just their 
first Job? Can they effectively communicate their ideas to multiple audiences in the 
many modes they need to? 

• Interdisciplinary Respect: Do your graduates understand the value of skills and 
perspectives from individuals in fields other than engineering? Do they respect both 
other fields and the individuals who practice in these fields? Are they able to work 
with these individuals? 

• Meet Grand Challenges: How prepared are your graduates to take on the wide range 
of roles— in government, industry, and academia— required for engineers to address 
the grand challenges that face the globe and its inhabitants? 


D.6 Welcoming Students into the Work World 

• Practicing Engineering: What challenges do your newly hired engineering graduates 
face when they begin a Job? What can you do to help facilitate their transition? Are 
they supported when they need to seek out information and advice? Are they given 
appropriate orientation, support and mentoring from others in the organization? 

• Working in Diverse Teams: Are the new hires able to work with a wide variety of 
coworkers and customers or clients in different roles and settings? Do they 
understand the value of skills and perspectives from individuals in fieids other than 
engineering? Do they understand that decisions can often incorporate more factors 
than those that pertain only to the engineering aspects? 

• Communicating Effectively: Do the new hires have an appreciation for the needs of 
different audiences when talking about their work or a problem? Are they able to 
listen to others and effectively incorporate input? Can they communicate their ideas 
to multiple audiences in the many modes they need to? 


Appendix E: Looking Ahead 


207 


E Appendix: 

Looking Ahead: Ideas for Future Research 


This appendix consists of sets of questions intended to stimuiate conversation about future 
work in the fieid of engineering education. The questions cover seven categories: student 
pathways, iearning engineering, significant iearning experiences, engineering knowing, 
teaching engineering students, researching issues in engineering education, and bringing 
about change in engineering education. 

The questions within each category were assembled with the following principles In mind: 

1. In each category, we start with questions that are directly grounded In CAEE research 
results. In some of the categories, later questions introduce broader issues that 
have arisen during the course of our research on the student learning experience, the 
faculty teaching experience, and the community of scholars In engineering education. 

2. We intentionally produced sets of questions that are extensive and far reaching. This 
is in keeping with our goal of stimulating new ideas and debate in the growing, 
interdisciplinary community that is engaged in scholarship in engineering education. 

3. We include questions that are at multiple levels of detail. In order to satisfy the 
Interests of a wide range of readers. 

4. Each category includes many types of questions. Some of the questions seek 
understanding of engineering students, engineering educators, and engineering 
researchers on dimensions examined in our existing work. Others concern 
explanations for specific CAEE findings, possible interventions, and more overarching 
philosophical issues. 


E.l On Pathways 

• What are the effects of institution, curriculum, student characteristics, and motivation 
on engineering persistence, migration, and career decision-making? 

• What characterizes informed decision-making with respect to a student’s choice to 
major in engineering? What factors are considered? What trade-offs are made? How 
do these factors and trade-offs differ for students with different backgrounds? Do 
they differ for students at various points in their pathways to an undergraduate 
degree? 

• What are the barriers to migrating into colleges of engineering? What are the 
“gatekeeper” courses (e.g., math)? 

• What core elements of engineering should P-12 students learn so that they can come 
to college prepared to enter engineering majors? What aspects of engineering should 
be taught at what grade levels In the P-12 system? 

• Thinking about the pre-college setting, what activities and experiences promote 
middle and high school students’ Interest in engineering? To what extent must these 
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activities and experiences create both intrinsic and extrinsic motivation to study 
engineering to be effective? 

• How can we best support students when they question whether to remain in 
engineering majors? 

• With an already crowded undergraduate curriculum, how can we help students catch 
up, if they did not take enough math and science in high school? 

• Why do students become less involved in their courses as seniors? 

• How can our understanding of changes in student engagement over time inform 
efforts to increase student engagement? What accounts for differences in levels of 
engagement among students? 

• What is it about interaction with faculty and involvement in engineering extracurricular 
activities that inspires and sustains students’ passion for engineering? How can we 
better capitalize on the motivating features of both formal and informal learning 
experiences? 

• How do financial challenges affect students’ undergraduate experiences? 

• What characterizes informed decision-making with respect to a student’s choice to 
pursue post-graduate work or graduate school? What factors are considered? What 
trade-offs are made? How do these factors and trade-offs differ for students with 
different backgrounds or at various points in their educational pathways? 

• In what ways do graduating students differentiate between the notions of a first Job 
and a long-term career when they are making future plans? Are they better prepared 
to consider one over the other? 

• For students who have both engineering and non-engineering career plans, what 
accounts for the breadth of their plans? E.g., do they have a specific, long-term vision 
of a professional pathway combining engineering and non-engineering pursuits, or are 
they uncertain of what opportunities will come their way? 

• How many students attend engineering school as a stepping stone to non-engineering 
careers? What do they see as the advantage of an engineering undergraduate 
degree? 

• How does parental background affect interest in engineering employment and 
graduate study? Does having parents with an engineering background affect a 
student’s choice to follow a particular academic pathway? Is there a relationship 
between socioeconomic status and interest in engineering? 

• How many students who express interest in engineering graduate school as 
graduating seniors attend graduate school at some point in their career? What 
factors affect their interest? What are the significant supports and barriers associated 
with attending graduate school? 

• How do early-career professionals and new graduate students reflect on and interpret 
their undergraduate experiences? In what ways do their undergraduate experiences 
help (or hinder) them in their new pursuits? 

• What are the lessons learned from students’ transitions to the workplace and 
graduate school? How do these lessons vary with industry, graduate school, size of 
organization, etc.? How can these lessons be fed back into and inform undergraduate 
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education? How can they be fed forward to graduate schools and engineering firms to 
help students’ make a smoother transition? 

• As graduates look back on their undergraduate experiences, what elements of their 
experiences do they see as most valuable? How do these elements compare with the 
ones they thought were important when they were in school? 

• What aspects of students and their educational experiences exhibit the most variation 
(e.g., in student background, interests, post-graduation plans)? 

• How do students’ pathways vary with gender and race/ethnicity? In what ways do 
underrepresented students’ college experiences differ from those of majority 
students? Even in cases where the experiences are ostensibly the same, to what 
extent are underrepresented and majority students impacted differently by the 
experiences? 


E.2 On Learning Engineering 

• What engineering concepts are most difficult for students to learn and why? What 
makes some concepts more difficult to learn than others? How can we better teach 
difficult concepts? 

• How can engineering design be integrated into more engineering learning 
experiences? 

• How can students be taught to better consider context in engineering design? How 
can students be taught to better take into account users and other important 
stakeholders in engineering design? 

• What life experiences can contribute to students’ consideration of context in 
engineering design? Are these experiences correlated with gender, race/ethnicity, 
socioeconomic status, and/or other demographic factors? 

• How can we help students better understand the integrated, interdisciplinary nature of 
engineering? 

• To what extent and in what ways do engineering students exhibit interdisciplinary 
respect (/.e., respect for other disciplines, individually and in combination with each 
other or with engineering)? How do engineering students compare with students in 
other disciplines in terms of interdisciplinary respect? How does entering the 
workplace affect interdisciplinary respect? How can we better educate engineering 
students to respect the knowledge and contributions of people from other disciplines? 

• How can engineering-focused educational activities be augmented to foster more 
development of professional skills? 

• In what specific ways is reported family income related to confidence in professional 
and interpersonal skills? 

• How can we help improve engineering undergraduates’ listening ability (in addition to 
their speaking and writing skills)? 

• How can students’ conative capacity, self-efficacy, and locus of control be leveraged to 
enhance student learning? How do they interact with one another to affect overall 
student success in college? 
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• What helps students recognize the areas in which they need more knowledge or 
skills? How can students learn how to seek out and acquire the knowledge and skills 
they need? 

• How do we foster self-directed learning in our students? 

• In what ways can reflecting on educational experiences enhance students’ intellectual 
and professional growth? How can we help students learn and practice this kind of 
metacognitive reflection? 

• How do we foster lifelong learning in our students? 

• How does the relationship between confidence and competence vary across student 
populations, across skill sets, and within students over time (/.e., during the course of 
their undergraduate years)? What strategies can be used to help students more 
accurately assess their competence, such that competence and confidence are 
aligned? 

• How does student identification with engineering (/.e., having an engineering identity) 
vary across students and within students over time? How does having strong 
identification with engineering support student success in engineering educational 
activities? Are there ways in which having a strong sense of identification with 
engineering can hinder students? 


E.3 On Significant Learning Experiences 

• How do students decide which extracurricular activities to be involved in? How do 
students benefit from these activities, both engineering-related and non-engineering- 
related? How can engineering programs support students to get the most benefit 
from their extracurricular experiences? 

o Why are co-ops, internships, and research experiences so often reported by 
students as significant learning experiences? 

o Is there a way to bring the most important aspects of those experiences into the 
classroom? 

o How can we help students link what they are learning in these experiences to what 
they learn in the classroom? 

o How can we ensure that co-ops, internships, and research experiences lead to 
significant engineering learning? 

• Why does students’ perception of the importance of professional and interpersonal 
skills remain unchanged during the undergraduate years, even though they are 
participating in more activities requiring these skills? 

• Why are engineering students less likely than students in other majors to take 
advantage of international learning experiences? Are the primary reasons lack of time 
and schedule flexibility? What do the students who go on these experiences gain? 
Given the global nature of contemporary engineering work and professional contexts, 
how can we better facilitate students’ international learning experiences? How can 
engineering programs make a more concerted effort to include these kinds of 
experiences? Is there a way to design local (/.e., on-campus) learning experiences that 
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teach some of the same skills and concepts that students learn when they travel 
overseas? 

• Why do relatively few engineering students take advantage of extracurricular 
community service and other social good project opportunities? What do the students 
who engage in these experiences learn? How do we reconcile relatively low 
involvement in these activities with the level of interest in making a difference in the 
world that engineering students report? 

• How can we better support students’ participation in service-learning curricular 
experiences? What do the students who engage in these experiences learn? 

• Why is it that school is not identified by more students as a source of their learning 
about the engineering profession? What does this indicate about engineering 
education? 


E.4 On Engineering Knowing 

• What math knowledge and skills are needed at what points throughout the 
engineering curriculum? How might better alignment of prerequisite courses with 
needed knowledge and skills affect recruitment of students into engineering? 

• What are the fundamental concepts that are common to multiple engineering 
disciplines? What is the “minimum set” of skills and concepts necessary for 
engineering practice? How do the increasing complexity and scale of engineering 
problems affect what we consider to be the “base” of engineering knowledge? 

• What aspects of engineering are important for all undergraduates to understand, 
regardless of their major? 

• In addition to becoming a practicing engineer, what should an engineering education 
be preparing students for? 

• How do we help students understand and reconcile the sometimes competing signals 
of engineering as a job (do what your boss says) and engineering as a profession (do 
what is good for society)? How do we help students identify and incorporate ideas 
about social consequences in their engineering design activities? 

• To what extent should we characterize engineering expertise in a holistic manner, e.g., 
by considering an engineers’ habits of practice, value systems, life experiences, 
ethics? (We might call this concept “engineering wisdom.”) What aspects of 
engineering wisdom could be taught during the undergraduate years? 

• How might an understanding of engineering concepts and approaches enable the 
general public to be more informed participants in a democratic society? 


E.5 On Teaching Engineering Students 

• How do engineering educators make significant decisions about the educational 
experiences they design? 
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• How can we support faculty in understanding variability in the classroom {e.g., in 
student background, interests, post-graduation plans) and how to use it to enhance 
teaching? 

• A thousand students, a thousand stories: How can the community better understand 
the aspects of students’ lives that contribute to important differences in their 
educational experiences? 

• In what ways can a structured examination of decision-making guide faculty with their 
teaching? 

• What are the most effective representations and methods for conveying engineering 
education research findings to engineering educators (e.g., academic papers, less 
formal written descriptions, visual representations, vignettes, personas)? 

• In what ways can engineering curricula be restructured to promote persistence, 
migration, and preparation for engineering workplaces? 

• How can APS findings about students’ motivation to study engineering, including their 
interest in effecting social good, inform the design of teaching innovations? 

• How can engineering educators address aspects of engineering practice that fall 
outside traditional engineering learning outcomes (e.g., tolerance for ambiguity, 
excitement about engineering, issues of identity, engineering failures, metacognition, 
skills of flexibility, and adaptability)? 

• What curricular methods and practices promote retention and application of 
engineering concepts across the curriculum, particularly in solving ambiguous, 
dynamic, real-life engineering problems? 

• What aspects of an engineer’s education are best served by campus-based 
experiences? By industry-based experiences? By technology-enhanced experiences? 
By service-learning experiences? 

• How can engineering expertise be characterized in a way that educators can use to 
facilitate more effective teaching? 

• What apprenticeship models of teaching can be employed to teach key elements of 
engineering practice? 

• What assessment methods are most effective for measuring aspects of engineering 
learning? 

• What tools and practices can we use to better assess the meaning and significance of 
students’ learning experiences from the perspective of the students themselves? 

• How can we take advantage of innovative assessment approaches and tools such as 
electronic learning portfolios to allow students to create richer and more holistic 
representations of the experiences that contribute to their becoming engineers? 

• How can these learner-centered approaches to assessment be used to better inform 
faculty, departments, and institutions, as well as graduate schools and employers, 
about students’ formal and informal experiences in engineering education? 

• How is teaching engineering the same as and different from teaching math, physical 
sciences, biological sciences, social sciences, the humanities, and the arts? 
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• How can new faculty be better supported in aligning classroom innovations with 
engineering education research? 

• How can engineering learning be better supported by industry? Are there “best 
practices” within and/or across companies that could be incorporated in 
undergraduate engineering education? What is the common thinking/wisdom, if any, 
about mentoring new engineers? 


E.6 On Researching Issues in Engineering Education 

• How can we support engineering facuity who want to pursue engineering education 
research? 

• In what ways do the process and experience of conducting research on learning and 
teaching change how an educator designs learning experiences? 

• What types of standards are currentiy used for evaluating scholarly contributions in 
engineering education? How do these standards influence the nature of the research 
that takes place (e.g., in terms of topics and research design)? 

• What types of support would help engineering education researchers create and 
manage interdisciplinary research teams? 


E.7 On Bringing About Change in Engineering Education 

• How can the extensive variabiiity of engineering students and their pathways through 
their educations be ieveraged in service of improving engineering education? 

• How do we support faculty and program planners in effecting change? 

• How do we support engineering educators in engaging in critical reflection on their 
own teaching? 

• What technological infrastructure (e.g., databases, tools) would accelerate both 
research advances and the rate at which research resuits are used to influence 
educational practice and outcomes? 

• How do innovations get promoted and adopted across institutions? What is the 
adoption pattern and history that past innovations have traced before becoming 
“common practice,” e.g., in the form of ABET requirements? 

• What theories of change are most appropriate for what campus/department cultures? 

• What critical configurations of circumstances or environments (/.e., tipping points) are 
more likely to bring about change? 

• How can deans and other administrators promote innovation in engineering 
education? What approaches have been successfui in achieving buy-in from faculty 
and support from university-level administration? 

• How can we best learn from specific, individuai success stories and generaiize from 
them to effect iarger-scaie change? 




201011021420 


information 


nialiingholn^“PP®™'' perspectives c»™“"":ate I 

,jnelp»«„„eerstaidinjfpp|“'Z^^^ 


wide 


multiple 

available 


understanding fppl 

I ■ A BA trancitinn I V d 


SyinStltUtlOnrirgineers?»f^^^^^^^^^ 

ciirnMioateneedH 


think 


know^hlo 

W nniiroiio international ^ institutional i /1 1 1 1 

I context ■ improvement prepareil” “ " ” 

■ — — — — ■ navigate 


academia 




earn ns 


Mwor 


ifacuItVh 

It pathway ^ |J 


a 


administrators range 

j advice 
modes 


qlTestions significant pljnSps 


might fjrrt teaching 
identity take'"^”®'* 


passion 


This material is based upon work supported by the National 
Science Foundation under Grant No. ESI-0227558. Any opinions, 
findings and conclusions or recommendations expressed in this 
material are those of the author(s) and do not necessarily reflect 
the views of the National Science Foundation (NSF). 


